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Editorials 


Welcome  to  the  third  volume  of  Annual  Review  of  CyberTherapy  and  Telemedicine.  A  decade  ago, 
CyberTherapy,  then  still  in  its  infancy,  only  existed  as  a  specialized  Virtual  Reality  and  Behavioral 
Healthcare  Symposium  at  the  Medicine  Meets  Virtual  Reality  (MMVR)  Conference.  At  that  first  ses¬ 
sion,  Dr.  Ralph  Lamson  presented  a  small  study  on  virtual  reality  (VR)  exposure  for  treatment  of  ac¬ 
rophobia  in  ten  participants.  It  is  now  clear  that  in  1996,  we  had  only  begun  to  realize  what  promise 
might  lie  ahead  for  both  VR  technology  and  the  CyberTherapy  Conference. 

Much  has  changed  over  the  past  ten  years  and  we  have  now  tapped  further  into  VR’s  potential  than 
many  of  us  could  have  ever  imagined.  Today,  researchers  from  around  the  world  are  busy  complet¬ 
ing  hundreds  of  trials,  applying  VR  for  such  varied  disorders  as  Anxiety,  Eating  Disorders  and  Obe¬ 
sity,  Addictions,  Erectile  Dysfunction,  Autism  and  Schizophrenia.  In  addition,  VR  for  Neurorehabiiita- 
tion  and  Physical  Rehabilitation  has  shown  definite  success,  as  has  VR  for  other  such  diverse  areas 
as  Pain  Distraction  (both  acute  and  chronic),  Education,  Training,  and  Physical  Disabilities. 

I  am  proud  to  report  that  as  VR’s  use  in  Behavioral  Healthcare  has  grown,  so  has  the  CyberTherapy 
Conference.  What  began  as  a  specialized  symposium  at  MMVR  concerned  mainly  with  conceptual 
matters,  has  now  grown  into  the  largest  program  on  controlled  clinical  trials  of  VR  and  other  ad¬ 
vanced  technologies  in  the  areas  of  behavioral  healthcare,  rehabilitation,  disabilities,  education,  and 
training. 

Along  with  my  colleagues  in  the  VR  community,  the  conference’s  focus  has  expanded  from  simply 
VR  to  now  include  such  cutting-edge  technologies  as  robotics,  non-invasive  physiological  monitoring, 
videogames,  E-health,  and  adaptive  displays.  CyberTherapy  2005  (CTOS)  represents  a  group  of  out¬ 
standing  international  researchers  and  clinicians  working  tirelessly  to  improve  our  understanding  of 
how  advanced  technologies  can  improve  21®'  century  healthcare.  In  addition  to  bringing  together  the 
best  clinicians  and  researchers  from  various  disciplines,  it  is  our  specific  intention  to  also  facilitate 
their  introduction  to  representatives  from  funding  agencies  interested  in  providing  support  for  ad¬ 
vanced  technologies  and  healthcare. 

In  celebration  of  a  decade  of  CyberTherapy,  this  year’s  Annual  Review  encompasses  a  state-of-the- 
art  collection  of  clinical  trials  with  an  eye  on  the  future  of  advanced  technologies  for  health  care.  In 
this  way,  I  hope  that  this  collection  can  serve  as  a  catalyst  to  improve  the  quality  of  life  of  the  patients 
we  seek  to  serve,  and  ultimately  allow  society  to  benefit  from  the  remarkable  technological  revolution 
that  is  occurring. 

I  hope  you  find  this  volume  to  be  both  an  exciting  and  useful  addition  to  your  bookshelf. 

Sincerely, 

Brenda  K.  Wiederhold,  Ph.D.,  MBA,  BCIA 
Co-Editor-in-Chief 
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Editorials 


Since  1986,  when  Jaron  Lamier  used  the  term  for  the  first  time,  virtual  reality  (VR)  has  been  usually 
described  as  a  collection  of  technological  devices:  a  computer  capable  of  interactive  three- 
dimensional  (3D)  visualization,  a  head-mounted  display,  and  data  gloves  equipped  with  one  or  more 
position  trackers.  The  trackers  sense  the  position  and  orientation  of  the  user,  and  report  that  informa¬ 
tion  to  the  computer  that  updates  (in  real  time)  the  images  for  display. 

In  medicine  many  researchers  share  this  vision:  VR  is  a  collection  of  technologies  that  allow  people 
to  interact  efficiently  with  3D  computerized  databases  in  real  time  using  their  natural  senses  and 
skills. 

However,  if  we  shift  our  attention  to  behavioral  sciences,  we  find  a  different  vision:  VR  is  described 
as  an  advanced  form  of  human-computer  interface  that  allows  the  user  to  interact  with,  and  become 
immersed  in  a  computer-generated  environment  in  a  naturalistic  fashion. 

In  fact,  psychologists  use  specialized  technologies  -  head-mounted  displays,  tracking  systems,  ear¬ 
phones,  gloves,  and  sometimes  haptic  feedback  -  to  develop  and  provide  a  new  human-computer 
interaction  paradigm.  In  VR,  patients  are  no  longer  simply  external  observers  of  images  on  a  com¬ 
puter  screen,  but  are  active  participants  within  a  computer-generated  3D  virtual  world.  This  approach 
is  clearly  detailed  in  many  papers  of  this  issue:  VR  is  used  as  an  advanced  communication  interface 
based  on  interactive  3D  visualization,  able  to  collect  and  integrate  different  inputs  and  data  sets  in  a 
single  real-like  experience. 

This  point  underlines  the  importance  of  the  therapeutical  relationship  for  the  efficacy  of  most  of  the 
approached  presented:  VR  is  a  tool  augmenting  the  possibility  of  reiation  and  interaction  between  the 
therapist  and  the  patient.  However,  many  psychotherapists  still  think  that  in  technology  supported 
treatments,  and  in  particular  in  VR  therapy,  there  is  no  place  for  a  therapeutic  relationship. 

To  allow  a  widespread  diffusion  of  communication  technologies  real  clinical  practice  is  critical  both  to 
counter  this  prejudice,  and  to  offer  training  courses  helping  therapists  in  understanding  how  to  inte¬ 
grate  technologies  in  their  own  approach  and  methods. 


Giuseppe  Riva,  Ph.D.,  M.S.,  M.A. 
Co-Editor-in-Chief 
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In  2005,  It  has  been  my  pleasure  not  only  to  co-organize  the  first  of  what  promises  to  be  a  long  line 
of  internationally  held  CyberTherapy  Conferences  here  in  Basel,  Switzerland,  but  also  to  help  cele¬ 
brate  the  conference’s  tenth  anniversary!  Once  again,  I  would  like  to  use  this  opportunity  to  I  extend 
a  warm  welcome,  or  as  we  say  here  in  Switzerland,  a  warm  “Grtizi”,  to  all  attendants. 

As  co-organizer  of  this  year’s  meeting,  it  has  also  been  my  pleasure  to  serve  as  one  of  the  Annual 
Review  of  Cybertherapy  and  Telemedicine’s  esteemed  board  of  editors.  This  year’s  Cybertherapy 
was  organized  as  a  joint  conference  together  with  the  1st  International  Conference  on  Applied  Tech¬ 
nologies  in  Medicine  and  Neuroscience  and  I  expect  a  fruitful  interdisciplinary  gathering  of  many  in¬ 
terested  researchers,  clinicians  and  engineers  alike.  I  do  believe  that  the  change  in  venue  for  the 
meeting  has  served  to  attract  an  even  more  diverse  number  of  researchers  from  around  the  world 
and  led  to  a  most  impressive  number  of  submissions  for  this  year’s  Annual  Review. 

I  would  like  to  thank  especially  Brenda  K.  Wiederhold  and  the  Interactive  Media  Institute  for  making 
this  joint  conference  possible.  Thanks  also  to  Stephane  Bouchard  who  with  utmost  efficiency  organ¬ 
ized  the  workshops.  And  finally,  a  special  thanks  goes  to  all  those  who  submitted  papers  and  post¬ 
ers,  to  all  participants  as  well  as  to  the  members  of  the  program  committee. 

We  all  hope  for  an  exciting  conference  that  all  of  you  will  enjoy  attending! 

Sincerely, 

Dr.  Alex  H.  Bullinger,  MBA 
Co-Editor-in-Chief 
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Critical  Reviews 


Motivational  Training  Tool  to  Promote  Healthy  Life  Styles 

N.  Fernandez  M.Sc.,  E.  del  Hoyo  Barbolla  M.Sc.,  C.  Ramirez  M.Sc.,  R.  de  las  Heras 
M.Sc.,  M.  T.  Arredondo  F.P.,  Ph.D. 

Life  Supporting  Technologies,  Telecommunication  Engineering  School,  Technical  University 

of  Madrid,  Spain 


Abstract:  The  aim  of  the  project  is  to  apply  augmented-reality  technology  to  teach  motor  skills  to 
patients  suffering  from  stroke.  To  achieve  this  goal  the  project  adopts  an  innovative  approach  based 
on  the  use  of  so-called  “motor  imagery.  ”  Recent  studies  in  neuroscience  have  provided  converging 
evidence  that  “imagining”  a  motor  action  invoives  the  same  brain  areas  involved  in  performing  the 
action.  This  supports  the  idea  -  already  exploited  by  sports  trainers  -  that  training  v/ith  motor  imagery 
(mental  practice)  couid  be  effective  in  learning  new  motor  skills  or  In  promoting  motor  recovery  after 
damage  to  the  central  nervous  system.  Previous  clinical  studies  have  shown  that  the  rehabilitation  of 
post-stroke  hemiplegic  patients  can  be  made  more  effective  by  combining  physical  practice  with 
mental  practice.  However,  for  many  patients  who  suffer  from  damage  to  the  central  nervous  system, 
mental  simulation  of  movements  can  be  difficuit,  even  when  the  relevant  neural  circuitry  has  not  been 
injured.  Starting  from  these  premises,  we  have  designed  and  developed  an  augmented-reality  work¬ 
bench  (called  “VR  Mirror”)  to  help  post-stroke  hemiplegic  patients  evoking  motor  images.  First,  the 
movement  is  acquired  by  the  system  from  the  healthy  arm.  Second,  the  movement  is  being  mirrored 
and  dispiayed  so  that  the  patient  can  observe  and  see  as  if  the  impaired  arm  is  performing  the  move¬ 
ment.  Third,  the  patient  is  instructed  to  rehearse  in  his/her  imagination  the  movement  he/she  has  just 
observed.  Last,  the  patient  has  to  perform  the  movement  with  the  affected  arm.  In  this  article,  we  de¬ 
scribe  preliminary  results  of  a  pilot  clinical  study,  which  has  evaluated  the  feasibility  of  using  technol¬ 
ogy-supported  mental  simulation  as  an  adjunct  in  the  rehabilitation  of  the  upper  limb  following  stroke. 
DESIGN:  Single-case  study.  SETTING:  Physical  Rehabilitation  Unit  of  Padua  Teaching  Hospital,  Pa¬ 
dua,  Italy.  SUBJECTS:  A  46-year-old  man  with  stable  motor  deficit  of  the  upper  right  limb  following 
stroke.  INTERVENTION:  The  patient  underwent  a  single-case  design,  with  four  weeks  intervention. 
The  intervention  consisted  of  3  practice  sessions  per  week  at  the  hospital  using  the  VR-Mirror,  in  ad¬ 
dition  to  usual  therapy  (Bobath  method).  This  intervention  was  followed  by  1-month  home- 
rehabilitation  program  using  a  portabie  device.  MAIN  OUTCOME  MEASURES:  The  patient  was 
evaluated  for  level  of  impairment  and  disability.  Pre-treatment  and  post  treatment  measures  included: 
the  upper-extremity  scale  of  the  Fugl-Meyer  Assessment  of  Sensorimotor  Impairment  and  the  Action 
Research  Arm  Test.  RESULTS:  The  patient  showed  improvement  in  upper  limb  score  as  measured 
by  the  two  scales.  CONCLUSIONS:  The  improvement  observed  in  the  patient  is  encouraging  and 
warrants  further  study. 


INTRODUCTION 

One  of  the  main  challenges  that  developed  so¬ 
cieties  face  is  the  prevention  of  non- 
communicable  diseases  such  as  diabetes  or 
obesity.  These  diseases  are  responsible  for 
60%  of  the  registered  global  deaths  and  for 
45%  of  the  global  burden  of  disease.  A  few, 
largely  preventable,  risk  factors  account  for 
most  of  the  world’s  disease  burden.  These  are 
high  cholesterol,  high  blood  pressure,  obesity, 
smoking  and  alcohol.  Chronic  diseases  are  the 


major  cause  of  death  and  disability  worldwide, 
and  increasingly  affect  people  from  countries  all 
over  the  world.® 

Guidelines  on  diet,  physical  activity  and  health 
serve  two  important  purposes:  to  guide  policy 
makers  and  to  educate  consumers,  about 
healthful  lifestyles.  In  order  to  promote  preven¬ 
tion  and  create  motivation  and  adherence  to  the 
prevention  programmes  it  is  highly  recom- 
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mended  to  stress  the  importance  of  lifestyle- 
health  relationship.  And  it  is  also  essential  to 
analyse  in  depth  how  to  formulate  these  pro¬ 
grammes  and  what  the  process  and  information 
they  should  convey.  The  strategies  drafted  by 
the  international  institutions  competent  in  this 
matter  stress  the  importance  to  increase  the 
overall  awareness  and  understanding  of  the 
influences  of  diet  and  physical  activity  on  health 
and  of  the  positive  impact  of  preventive  inter¬ 
ventions. 

The  work  presented  in  this  paper  contributes 
partially  to  solve  this  situation  by  using  one  of 
the  most  effective  strategies:  the  prevention, 
through  the  procedure  of  education.  In  this 
sense,  providing  users  with  personalised  infor¬ 
mation  is  key  to  be  able  to  adhere  them  to  the 
prevention  strategy.  More  than  that,  the  new 
healthcare  delivery  process  models  tend  to  be 
citizen/patient  centred,  that  are  respectful  of  the 
individual  preferences,  needs  and  values  and 
place  information  and  training  at  the  heart  of  the 
healthcare  delivery  strategies.^ 

An  interactive  and  visual  tool  based  on  virtual 
3D  technologies  for  the  Internet,®  using  X3D,  a 
powerful  and  extensible  open  file  format  stan¬ 
dard  for  3D  visual  effects,  has  been  developed 
and  integrated  in  an  e-learning  system  that  of¬ 
fers  personalised  information  about  healthy  life¬ 
styles. 

MATERIALS  AND  METHODS 

The  tool  implemented  intends  to  give  answers 
to  the  situation  referred  above  by  offering  the 
users  personalised  information  for  their  own 
self-care  and  moreover,  motivating  them  to 
make  use  of  this  information  and  take  control 
over  their  decisions  regarding  their  lifestyle. 
Personalised  information  makes  users  more 
independent,  makes  them  conscious  of  the  pos¬ 
sible  results  due  to  their  behaviour  and  facili¬ 
tates  them  to  take  care  of  their  own  health.  In 
this  sense,  the  development  of  a  virtual  reality 
tool  helps  to  motivate  the  user  and  also  im¬ 
proves  the  e-learning  process.^ 

The  main  contribution  of  the  implemented  sys¬ 
tem  is  the  delivery  of  personalised  information 
in  an  adapted  environment.  This  goal  has  been 
achieved  by  the  integration  of  several  technolo¬ 
gies  such  as  XML,  native  databases,  X3D, 


VRML  or  JSP  (Java  Server  Technology).  These 
technologies  have  been  chosen  taking  into  ac¬ 
count  the  interactive,  portable,  visual  and  inte¬ 
grative  identified  requirements  for  the  tool.  Fur¬ 
thermore,  these  technologies  allow  the  possibil¬ 
ity  of  introducing  some  kind  of  intelligence  to  the 
tool  behind  the  interface  in  order  to  obtain  a 
system  that  could  respond  to  the  individual  user 
needs  and  preferences. 

The  complete  process  has  been  divided  into 
four  different  stages.  Therefore,  in  each  step,  a 
group  of  technologies  has  been  selected  to 
achieve  the  required  functionalities  for  the 
whole  system.  These  stages  are:  Profiling,  Stor¬ 
age,  Personalisation  &  Management,  The  next 
paragraphs  describe  these  four  stages: 

The  Profiling  stage  helps  to  acquire  individual 
user  information,  to  structure  it  and  hence,  to 
define  what  the  user  needs  are.  The  system  is 
designed  to  be  accessed  through  the  Internet. 
The  registration  and  the  questionnaires  about 
nutritional,  statistical,  physical  activity  and  moti¬ 
vational  data  have  been  developed  with  JSP. 
The  goal  of  these  questions  is  to  define  a  user 
profile  and  to  discover  his  motivation  status  ac¬ 
cording  to  the  “States  of  Change”  methodology.® 
The  methodology  sets  the  characteristics  to  be 
classified  in  each  state  of  change  and  the  moti¬ 
vational  techniques  to  be  used  for  citizens. 

The  Storage  stage  responds  to  the  need  of 
storing  the  user  profiles  and  the  information  to 
be  shown  in  the  system.  Each  profile  is  stored 
as  an  XML  file  in  the  native  database.  The  sce¬ 
narios  have  been  developed  in  X3D.  This  allows 
describing  a  virtual  world  in  XML  by  means  of 
predefined  tags  representing  objects.  The  sce¬ 
narios  are  virtual  worlds  that  simulate  the  “real 
world”  and  daily  activities.  However,  as  the  X3D 
plug-in  was  not  compliant  with  the  X3D  stan¬ 
dard  at  the  development  time,  the  display  of  the 
sessions  is  performed  by  using  VRML  (an  X3D 
antecedent).  Translations  between  the  two  lan¬ 
guages  are  made  by  applying  a  style  sheet  to 
the  X3D  files  in  their  XML  format. 

The  Management  stage  is  necessary  in  order 
to  maintain  the  static  information  used  by  the 
system. 

Finally,  the  Personalisation  stage  carries  out 
the  process  of  adapting  the  system  to  the  users. 
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At  the  time  the  user  profile  is  created  or  modi¬ 
fied  (either  by  the  user  or  by  the  intelligence 
added  to  the  system),  the  scenarios  are  person¬ 
alised  to  meet  the  user  preferences  and  are 
completed  with  the  information  that  suits  the 
user  needs  most  and  they  are  stored  in  the  sys¬ 
tem.  These  tasks  involve  reading  and  writing 
XML  files  by  means  of  a  XML  parser.  It  is  used 
a  SAX  (Simple  API  for  XML)  parser  because  of 
its  speed  and  straightforward  features.  Once  the 
personalised  worlds  have  been  created,  they 
are  stored. 

RESULTS 

The  tool  implemented  has  been  developed  to 
present  the  following  features: 

•  To  offer  personalised  information  in 
form  and  content 

•  To  be  visual  and  interactive 

•  To  be  integrated  in  an  e-learning  sys¬ 
tem 

In  this  sense,  a  training  activity  in  the  system 
consists  of  an  Internet  multimedia  session  that 
shows  personalised  information  by  means  of  an 
attractive  visual  interface.  The  session  starts  by 
choosing  the  users  a  login  and  a  password  for 
identification  and  introducing  their  data.  The  user 
profiles  are  created  the  first  time  the  user  ac¬ 
cesses  the  system  by  filling  in  a  structured  ques¬ 
tionnaire  about  nutritional,  statistical,  motivational 
and  physical  exercise.  Each  group  of  questions 
has  a  specific  objective:  the  statistical  questions 
try  to  classify  the  users  per  age,  gender,  health 
status,  ICT  knowledge  or  profession.  Nutritional 
questions  try  to  enquire  if  the  users  are  vegetar¬ 
ian  or  if  it  is  common  that  they  often  eat  out  in 
order  to  prepare  a  customised  diet.  Motivational 
questions  are  related  to  the  importance  the  users 
give  to  their  self-care  through  the  lifestyle 
adopted  up  to  the  moment  or  the  intentions  to 
incorporate  healthier  attitudes  in  the  future  (i.e., 
intention  to  lose  weight,  to  practise  any  sport, 
etc). 

In  subsequent  accesses  the  users  must  enter 
the  login  and  password  to  be  recognised  by  the 
system  as  registered  users  and  this  way,  skip¬ 
ping  the  questionnaires  and  be  led  directly  to 
the  welcome  page  where  they  can  choose  ei¬ 


ther  starting  their  training  session  or  modifying 
their  data. 

The  personalisation  of  the  scenarios  involves 
presenting  different  information  (links,  articles, 
recipes,  sport  videos,  etc.)  to  each  user  profile. 
In  order  to  perform  the  personalisation,  the  sys¬ 
tem  accesses  the  database  and  extracts  the 
model  files  stored  in.  The  user  session  “world” 
is  then  created  from  this  pattern  and  is  filled  in 
according  to  the  user  needs.  Adapting  the  infor¬ 
mation  includes  not  only  the  content  but  also 
the  form  (i.e.  tone  in  which  it  is  presented,  ways 
in  which  the  information  is  shown  to  the  user). 
This  depends  strongly  on  the  user  motivation 
state"*  to  follow  advice  and  to  adopt  changes  to 
a  healthier  lifestyle  which  was  identified  in  the 
profiling  stage.  In  this  sense,  only  the  relevant 
information  in  the  appropriate  mode  (direct,  indi¬ 
rect,  soft,  etc.)  is  delivered  to  the  user  through  a 
scenario  personalised  accordingly  (avatar,  col¬ 
ours,  links,  etc.). 

The  database  has  been  hierarchically  structured 
in  three  collections  (similar  to  directories):  pro¬ 
files,  patterns  and  information.  The  most  impor¬ 
tant  is  the  first  of  them  and  it  is  due  to  be  dy¬ 
namically  changed  every  time  a  new  user  ac¬ 
cesses  the  system  by  storing  the  user  profile 
and  creating  a  user  collection  under  it.  In  the 
user  collection,  called  as  the  user  login,  the  per¬ 
sonalised  scenarios  are  stored.  In  this  way,  if 
the  user  has  not  introduced  any  change  in  their 
data  from  the  previous  session,  the  system  ac¬ 
cesses  the  adapted  scenarios  and  displays 
them  to  the  user,  thus,  saving  time  and  re¬ 
sources.  The  other  collections,  patterns  and 
information,  contain  static  data  such  as  scenar¬ 
ios  models  or  information  to  be  shown  to  the 
user.  The  pattern  collection  stores  the  scenarios 
models  files.  Each  of  these  files  contains  the 
common  information,  which  is  shared  between 
all  the  users  in  a  specific  scenario,  such  as  im¬ 
ages  or  help  links.  The  second  collection 
(information)  stores  data  related  to  health  care 
in  different  modes,  depending  on  the  needs  the 
user  has.  The  corresponding  information  will  be 
selected  during  the  personalisation  stage  de¬ 
pending  on  the  user  needs  and  motivation. 

The  files  stored  in  these  collections  can  be 
modified  by  the  administrator  in  order  to  im¬ 
prove  the  system  quality,  by  adding  new  infor- 
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mation  or  modifying  the  existing  one;  adding 
new  features  to  the  scenarios  or  readapting  the 
ones  stored  in  previous  sessions.  It  is  important 
to  notice  that  changes  to  the  data  and  scenarios 
will  not  affect  the  users  due  to  the  models  us¬ 
age  to  generate  the  personalised  worlds.  In  this 
way,  the  next  session  the  users  establish  with 
the  system,  the  scenarios  (“worlds”)  shown  to 
them  will  include  the  changes  performed  and 
the  new  added  or  modified  information  if  it  is 
relevant  to  them. 

The  tool  is  completely  developed  and  personal¬ 
ization  features  are  included,  a  process  we 
started  working  together  with  healthcare  profes¬ 
sionals.  Different  users'  profiles  have  been 
identified. 

The  tool  has  been  tested  with  a  number  of  us¬ 
ers  to  check  whether  the  tool  and  the  informa¬ 
tion  provided  suited  the  users’  needs.  A  survey 
was  handed  out  for  completion  after  the  first 
training  session  in  order  to  measure  their  satis¬ 
faction  with  the  system.  The  survey  is  structured 
in  different  parts  which  are  related  to  the 
adaptability  of  the  contents,  the  appropriateness 
of  the  relation  between  the  scenarios  and  the 
information  showed  in  each  of  them,  the  choice 
of  Virtual  Reality  as  the  interface  and  the  usabil¬ 
ity  of  the  global  tool.  Statistical  data  were  also 
collected,  such  as  the  age,  the  job  or  the  com¬ 
puter  knowledge  level. 

The  results  of  the  survey  indicate  that  Virtual 
Reality  is  an  unknown  technology  yet  although  it 
presents  high  potential  for  the  implementation  of 
e-learning  systems.  Elderly  people  value  it  quite 
negatively  opposed  to  the  students  and  techni¬ 
cal  professionals.  A  similar  answer  is  also  given 
to  the  usability  of  the  global  tool  due  to  the 
strong  dependence  this  aspect  has  with  the 
ability  of  handling  Virtual  Reality.  However,  the 
users  state  that  the  relation  between  the  sce¬ 
narios  developed  and  the  information  provided 
in  each  of  them  was  tailored  to  their  needs  and 
the  contents  were  marked  as  a  good  educa¬ 
tional  tool  to  adopt  a  healthy  lifestyle. 

CONCLUSIONS 

An  interactive  and  visual  tool  has  been  devel¬ 
oped  and  integrated  in  an  e-learning  system 
that  offers  personalised  information  about 
healthy  lifestyles.  We  consider  that  this  is  a  very 


interesting  research  line  due  to  the  increase  of 
e-learning  systems  and  the  importance  of  hav¬ 
ing  access  to  quality  information  related  to 
health.  Joining  both  in  the  same  system  pro¬ 
vides  citizens  with  a  way  to  obtain  information  to 
possible  them  to  be  responsible  for  their  own 
health.  People  should  have  better  access  to 
trusted  sources  for  the  information  they  need, 
tailored  to  their  individual  requirements.  In  this 
way,  the  implemented  tool  goes  further  by  add¬ 
ing  a  motivational  feature  with  the  aim  of  mak¬ 
ing  users  feel  taking  control  of  their  own  self- 
care. 

To  conclude,  the  tool  developed  provides  certi¬ 
fied  information  and  achieves  personalisation  by 
considering  different  dimensions  of  people’s 
lifestyle  and  filtering  the  information  according 
to: 

•  Who  the  user  is;  a  Healthcare  professional, 
a  Citizen,  etc. 

•  What  the  user  needs;  different  information 
or  services  the  user  may  need 

•  Where  the  user  is:  considering  different  sce¬ 
narios,  including  “on  the  move” 

•  How  the  user  needs  the  service:  consider¬ 
ing  different  type  of  devices 

However,  the  tool  has  strong  requirements  to 
perform  in  an  acceptable  way.  In  this  sense,  it 
may  be  a  solution  to  try  to  adopt  more  dynamic 
software  in  order  to  be  able  to  include  it  in  other 
devices  such  as  mobile  phones  or  PDAs  that 
facilitate  the  information  access  in  active  life¬ 
styles. 
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Abstract:  Environments  used  up  to  now  for  therapeutic  applications  are  invariable  ones  whose  con¬ 
tent  cannot  be  changed  by  the  therapist  or  by  the  patient.  However,  this  is  a  technical  issue  that  can 
be  solved  with  current  technology.  In  this  paper,  we  describe  a  virtual  environment  that  has  been  de¬ 
veloped  taking  into  account  this  factor.  The  main  technical  feature  of  the  environment  is  that  its  as¬ 
pect  can  be  controlled  and  modified  by  the  therapist  that  conducts  the  clinical  sessions  depending  on 
the  emotions  that  the  patient  is  feeling  at  each  moment. 

The  applications  of  these  dynamic  changes  are  not  limited  to  the  field  of  clinical  psychology.  They 
open  a  new  arena  of  possibilities  for  many  other  kinds  of  applications  including  industry,  architecture, 
medicine  etc.  The  virtual  environment  that  is  described  in  this  paper  is  a  step  towards  a  truly  adaptive 
display. 


INTRODUCTION 

The  concept  of  “adaptive  displays”  has  been 
considered  for  many  years.  It  is  the  use  of  tech¬ 
nical  devices  that  adapt  to  the  requirements  of 
the  user,  rather  than  having  the  users  adapt  to 
the  device.  The  term  “adaptive  display”  has  re¬ 
ferred  to  displays  that  change  their  contents 
depending  only  on  situational  content  rather 
than  on  any  awareness  of  the  user.^  These 
adaptive  displays  can  be  used  in  many  different 
applications  such  as  medicine,  industry,  archi¬ 
tecture,  and  psychology. 

In  the  field  of  clinical  psychology,  virtual  reality 
has  in  recent  years  been  applied  to  the  treat¬ 
ment  of  psychological  disorders.  This  idea  was 
first  voiced  in  November  1992  in  the  Human- 
Computer  Interaction  Group  of  the  Clark  Atlanta 
University.  The  first  experiment  was  a  pilot 
study  with  a  32-years-old  woman  who  suffered 
from  flying  phobia.  She  was  treated  using  a 
virtual  environment^  in  which  she  followed  eight 
thirty-minute  sessions.  At  the  beginning  of  each 
session,  the  subject  had  a  high  level  of  anxiety, 
which  was  decreasing  gradually  after  remaining 
in  the  situation  for  several  minutes.  This  effect 
also  was  transferred  from  the  virtual  to  the  real 
world. 


Since  this  initial  study,  this  technique  has  been 
used  for  the  treatment  of  different  phobias.  We 
can  point  out  the  environments  designed  for 
acrophobia  treatment,^'®  agoraphobia,^^  spider 
phobia®  or  flying  phobia. Its  effectiveness 
has  been  analyzed  for  the  fight  against  other 
psychological  problems:  obsessive-compulsive 
disorders,  attention  deficit  disorders,  post- 
traumatic  stress  and  eating  disorders. 

However,  environments  used  up  to  now  for 
therapeutic  applications  are  invariable  ones 
whose  content  allows  for  only  minor  changes. 
As  a  result,  no  modification  can  be  made  in  the 
contents  of  the  virtual  environment  even  when 
high  emotional  responses  can  be  obtained  from 
the  user.  It  is  our  claim,  that  the  psychologist 
should  have  a  greater  control  over  the  aspect  of 
the  virtual  environment  that  is  shown  to  the 
user. 

In  the  environment  that  we  are  presenting  in 
this  paper,  the  psychologist  can  make  changes 
in  the  aspect  of  the  environment  depending  on 
the  reactions  of  the  patient  to  different  parts  of 
the  therapy  session.  This  work  has  been  con¬ 
ducted  inside  the  EMMA  Project  (IST-2001- 
39192). 
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DESIGN  OF  THE  VIRTUAL  ENVIRONMENT 


DESCRIPTION  OF  THE  ENVIRONMENT 


The  purpose  of  the  virtual  environment  is  to  be 
used  in  the  context  of  a  psychological  treat¬ 
ment.  The  user  is  a  person  who  suffers  from 
psychological  problems,  such  as  affective  disor¬ 
ders,  anxiety  disorders  or  adjustment  disorders. 
All  treatment  sessions  are  conducted  by  a 
therapist  and  the  virtual  environment  is  used  as 
an  aid  for  this  treatment.  Both  the  therapist  and 
the  patient  are  physically  present  in  the  room 
during  the  evolution  of  the  session. 

The  patient  visualizes  the  virtual  environment  in 
a  retro-projected  screen,  and  the  interaction  is 
made  by  means  of  a  wireless  joystick. 

The  application  has  been  developed  using 
Brainstorm  eStudio  software.  This  is  an  ad¬ 
vanced,  multiplatform  real  time  3D  graphics  de¬ 
velopment  tool.  It  incorporates  features  such  as: 
the  easy  inclusion  of  3D  objects  imported  from 
files,  erased  or  animated;  generation  of  3D 
texts,  whose  colors  can  be  changed;  adding 
videos  to  surfaces;  generation  of  new  interfaces 
and  timers;  addition  of  actions  to  objects  when 
they  are  selected;  addition  of  sounds;  loading  or 
saving  configurations.  All  these  possibilities  are 
controlled  using  the  mouse,  pop-up  menus  or 
drag  and  drop  actions.  The  interpreted  lan¬ 
guage  that  is  used  to  program  is  python.  Brain¬ 
storm  eStudio  can  be  defined  as  an  interface 
that  the  programmer  can  use  to  create  3D  com¬ 
plex  visualizations  using  only  tool  options. 


The  environment  can  be  modified  dynamically 
by  the  therapist  taking  into  account  the  state  of 
the  patient  at  each  moment  during  the  treat¬ 
ment,  so  different  aspects  have  been  devel¬ 
oped.  The  environment  is  not  a  static  one.  Ini¬ 
tially,  the  user  appears  inside  a  big  circular¬ 
shaped  hall  with  no  walls. 

The  user  can  visualize  the  outer  part  of  the  en¬ 
vironment,  which  initially  is  an  open  area  of 
meadows.  The  user  can  navigate  freely  along 
the  entire  environment,  even  leave  the  big  hail 
and  walk  through  the  meadows. 

However,  the  meadows  constitute  only  one  of 
the  five  possible  aspects  of  the  outer  part  of  the 
environment.  The  other  pre-defined  aspects 
are:  a  desert,  an  island,  a  forest  with  many  trees 
and  branches,  and  a  snowed  city.  These  five 
environments  can  be  related  to  different  emo¬ 
tions.  For  example,  the  forest  with  braches  can 
be  related  to  anxiety.  On  the  other  hand,  if  the 
purpose  of  the  therapist  is  to  induce  relaxation 
in  the  patient,  the  island  can  be  used.  The  de¬ 
sert  can  be  related  to  rage,  and  the  snowed  city 
with  sad  situations,  so  it  can  be  used  during  the 
session  when  the  patient  is  remembering  a 
situation  that  induces  sadness  in  him  or  her. 
The  use  of  each  aspect  of  the  environment  de¬ 
pends  on  the  context  of  the  session  and  can  be 
selected  by  the  therapist  in  real  time. 

In  order  to  control  the  appearance  of  the  differ¬ 
ent  aspects  that  have  been  developed,  a  spe- 


Figure  1.  Aspect  of  the  room. 
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Figure  3.  Interface  for  the  therapist 
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Figure  4.  The  sad  environment  snowing  and  with  a  storm. 


fects  can  be  applied  to  the  environment:  a  rain¬ 
bow  can  appear,  it  can  start  raining,  snowing, 
an  earthquake  can  be  generated,  and  the  illumi¬ 
nation  can  change  to  the  one  corresponding  to 
different  hours  of  the  day  and  the  night. 

All  these  effects  are  also  launched  from  the 
same  interface  that  controls  the  big-scale 
changes.  The  therapist  can  control  by  means  of 
a  slider  the  moment  when  the  effect  is  launched 
and  its  intensity. 

OTHER  INTERACTIONS 

But  it  is  not  only  the  therapist  who  can  control 
the  personalization  of  the  environment.  The 
user  can  also  play  an  active  role  on  this  task. 

The  big  hall  that  is  present  in  any  of  the  possi¬ 
ble  aspects  of  the  virtual  environment  is  com¬ 
posed  by  different  systems  that  interoperate 
and  allow  the  patient  to  express  ideas  using 
different  items  (videos,  images,  sounds,  colors 
and  3D  objects)  that  can  be  selected  from  a 
database. 

Distributed  along  the  environment,  there  are 
several  places  (object  holders)  where  the  pa¬ 
tient  can  locate  an  item  from  the  database,  so 
the  3D  aspect  of  the  element  will  be  shown,  or 


the  associated  sound  or  video  will  be  repro¬ 
duced.  This  can  be  used  as  a  way  to  personal¬ 
ize  the  environment.  These  object  holders  can 
serve  as  a  mixer  tool  to  combine  several  ele¬ 
ments  to  form  a  new  complex  element.  This  is 
achieved  if  different  elements  from  different 
categories  are  copied  to  the  same  object  holder. 
There  are  some  special  object  holders  that  are 
placed  in  a  balcony  of  the  room.  The  patient  can 
modify  the  size  of  the  objects  that  are  placed  on 
them  by  means  of  the  loudness  of  the  voice. 
They  will  be  used  as  a  discharge  area  where 
the  patient  can  give  free  rein  to  his/her  feelings. 
A  system  has  been  programmed  that  detects 
the  loudness  of  the  input  sounds  and  modifies 
accordingly  the  size  of  the  objects  placed  on 
those  special  object  holders. 

The  living  book  is  a  special  object  where  the 
patient  can  place  icons  that  represent  the  ele¬ 
ments  from  the  database.  These  icons  can  be 
classified  in  different  chapters.  It  is  the  instru¬ 
ment  that  the  patient  will  use  to  put  in  order  the 
contents  that  have  been  used  during  the  clinical 
session  with  the  therapist. 

Finally,  the  drain  can  be  used  to  destroy  any 
element  that  is  not  needed  anymore. 
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CONCLUSIONS 

Using  this  new  technical  approach,  different 
environments  can  be  developed  to  treat  differ¬ 
ent  kinds  of  psychological  problems.  The  possi¬ 
bility  of  dynamically  changing  the  aspect  of  the 
environment,  which  is  the  main  improvement  of 
this  system  from  the  technical  point  of  view, 
opens  a  new  area  and  new  possibilities  for 
treating  psychological  disorders. 

Besides,  this  system  constitutes  a  step  forward 
towards  a  true  adaptive  display.  The  ultimate 
purpose  would  be  that  the  system  itself  would 
be  able  to  detect  the  emotional  state  of  the  user 
and  adapt  its  contents  to  this  state  without  any 
intervention  from  the  therapist. 
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Abstract:  Presence  research  studies  the  experience  of  being  in  a  piace  or  being  with  someone  as  it 
is  mediated  through  technology.  The  experience  of  presence  appears  to  be  a  complex  perception, 
formed  through  an  interplay  of  raw  multisensory  data,  spatial  perception,  attention,  cognition,  and 
motor  action,  all  coupled  through  a  constant  dynamic  loop  of  sensorimotor  correspondence.  The  fact 
that  technology  can  start  working  as  a  transparent  extension  of  our  own  bodies  is  critically  dependent 
on  (i)  intuitive  interaction  devices  which  are  ‘invisible-in-use’,  seamlessly  matched  to  our  sensorimo¬ 
tor  abilities,  and  (ii)  the  highly  plastic  nature  of  our  brain,  which  is  continuously  able  and  prone  to 
adapt  to  altered  sensorimotor  contingencies.  The  perception  of  ourselves  as  part  of  an  environment, 
virtual  or  real,  critically  depends  on  the  ability  to  actively  explore  the  environment,  allowing  the  per¬ 
ceptual  systems  to  construct  a  spatial  map  based  on  sensorimotor  dependencies.  Provided  the  real¬ 
time,  reliable  correlations  between  motor  actions  and  multisensory  inputs  remain  intact,  the  integra¬ 
tion  of  telepresence  technologies  into  our  ongoing  perceptual-motor  loop  can  be  usefully  understood 
as  a  change  in  body  image  perception  -  a  phenomenal  extension  of  the  self 


INTRODUCTION 

Interactive  systems  that  allow  users  to  control 
and  manipulate  real-world  objects  within  a  re¬ 
mote  real  environment  are  known  as  teleopera¬ 
tor  systems.  Remote-controlled  manipulators 
(e.g.,  robot  arms)  and  vehicles  (e.g.,  NASA’s 
Mars  Exploration  Rovers)  are  being  employed  to 
enable  human  work  in  hazardous  or  challenging 
environments  such  as  space  exploration,  under¬ 
sea  operations,  or  hazardous  waste  clean-up. 
They  also  allow  for  transforming  the  temporal 
and  spatial  scale  of  operation,  as  is  the  case 
with  for  instance  minimally  invasive  surgery.  In 
teleoperation,  the  human  operator  directly  and 
continuously  guides  and  causes  each  change  in 
the  remote  manipulator.  Sensors  at  the  remote 
site  (e.g.,  a  stereoscopic  camera,  force  sensors) 
provide  continuous  feedback  about  the  slave's 
position  in  relation  to  the  remote  object,  thereby 
closing  the  continuous  perception-action  loop 
that  involves  the  operator,  the  master  system 
with  which  she  interacts  locally,  and  the  remote 
slave  system.  In  the  context  of  telerobotics, 
telepresence  is  closely  associated  to  the  sense 
of  distal  attribution,'  the  externalisation  of  the 
self  to  include  remote  tools  that  phenomenologi¬ 
cally  become  extensions  of  one's  own  body, 
even  if  they  are  not  physically  part  of  it. 

Whereas  teleoperation  systems  enable  the  ma¬ 
nipulation  of  remote  real-world  environments  and 


objects  within  it,  virtual  environments  (VEs)  allow 
users  to  interact  with  synthetic  or  computer¬ 
generated  environments.  In  its  most  well-known 
incarnation,  VEs  are  presented  to  the  user  via  a 
head-mounted  display  (HMD)  where  visual  infor¬ 
mation  is  presented  to  the  eyes  via  small  CRTs 
or  LCDs,  and  auditory  information  can  be  pre¬ 
sented  using  headphones.  Importantly,  the  HMD 
is  fitted  with  a  position  tracking  device  which  pro¬ 
vides  the  necessary  information  for  the  computer 
to  calculate  and  render  the  appropriate  visual 
and  auditory  perspective,  congruent  with  the 
user's  head  and  body  movements.  Haptic  infor¬ 
mation,  although  not  yet  usually  included  in  pre¬ 
sent-day  VEs,  can  be  added  through  the  use  of 
for  instance  an  exoskeletal  glove  or  arm,  acting 
both  as  sensor  and  actuator. 

Telepresence  (in  relation  to  teleoperation)  and 
virtual  presence  (in  relation  to  VEs)  both  address 
the  psychological  phenomenon  of  presence  - 
the  sense  of  'being  there’  in  a  mediated  environ¬ 
ment,  or  a  “perceptual  illusion  of  non-mediation” 
as  Lombard  and  Ditton^  defined  it.  Perceived 
transparency  of  the  medium  is  crucial,  i.e.  a 
sense  of  direct  perceptual  stimulation  and  poten¬ 
tial  for  action,  without  an  awareness  of  the  re¬ 
moteness  in  time  or  space  of  the  simulated  or 
reproduced  realities. 
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MEDIA  CHARACTERISTICS  INFLUENCING 
PRESENCE 

Since  the  early  1990s  onwards,  presence  has 
been  empirically  studied  in  relation  to  various 
media,  most  notably  VEs.  A  large  number  of  fac¬ 
tors  that  may  potentially  influence  the  sense  of 
presence  have  already  been  suggested  in  the 
literature.  Depending  on  the  levels  of  appropri¬ 
ate,  rich,  and  consistent  sensory  stimulation, 
varying  levels  of  presence  can  be  produced. 
Sheridan^  proposed  three  categories  of  determi¬ 
nants  of  presence; 

The  extent  of  sensory  information  presented  to 
the  participant,  i.e.  the  amount  of  salient 
sensory  information  presented  in  a  consis¬ 
tent  manner  to  the  appropriate  senses  of 
the  user. 

The  level  of  control  the  participant  has  over  the 
various  sensor  and  interface  mechanisms 
(tracked  HMD,  dataglove,  etc.).  This  refers 
to  the  various  sensorimotor  contingencies, 
i.e.  the  mapping  or  correlation  between  the 
user’s  actions  and  the  perceptible  spatio- 
temporal  effects  of  those  actions. 

The  participant's  ability  to  modify  the  environ¬ 
ment,  i.e.,  the  ability  to  interact  with  the  vir¬ 
tual  or  remote  environment  and  to  affect  a 
change  within  that  environment. 

The  extent  of  sensory  information  is  similar  to 
Steuer’s'*  notion  of  vividness.  Level  of  control 
over  sensors  and  effectors,  and  the  ability  to 
modify  the  environment  correspond  to  Steuer's'* 
notion  of  interactivity.  These  three  factors  all 
refer  to  the  media  form,  that  is,  to  the  physical, 
objective  properties  of  the  media  technology. 
Importantly,  presence  research  is  about  relating 
those  media  form  variables  to  the  human  re¬ 
sponse.  In  principle.  Slater's®  contention  that  one 
should  try  to  keep  content  variables  out  of  the 
presence  equation  as  much  as  possible  is 
adopted  here.  According  to  Slater,®  “presence  is 
about  form,  the  extent  to  which  the  unification  of 
simulated  sensory  data  and  perceptual  process¬ 
ing  produces  a  coherent  'place'  that  you  are  ‘in’ 
and  in  which  there  may  be  the  potential  for  you 
to  act.” 

It  is  of  clear  theoretical  and  practical  value  to 
establish  what  the  optimal  mix  of  cues  might  be 
for  different  application  contexts,  or,  if  the  opti¬ 


mum  is  unattainable,  which  elements  are  most 
critical  to  the  experience  of  presence.  It  appears, 
for  instance,  that  pictorial  realism  contributes 
less  to  the  experience  of  presence  in  VEs  than, 
for  instance,  interactivity  of  viewpoint.®  As 
Heeter^  noted,  “the  alchemy  of  presence  in  VR  is 
in  part  a  science  of  tradeoffs.”  It  is  not  clear  at 
present  how  much  each  feature  or  perceptual 
cue  contributes  to  eliciting  a  sense  of  presence 
for  the  participant  (i.e.,  the  relative  weighting), 
nor  is  it  clear  how  these  cues  interact  with  each 
other.  A  model  of  multisensory  information  inte¬ 
gration  and  interaction  for  presence  -  a  presence 
equation,  if  you  like  -  will  be  a  valuable  theoreti¬ 
cal  and  practical  contribution.  Ellis®  has  argued 
that  an  equation  relating  presence  to  its  contrib¬ 
uting  factors  should  allow  for  iso-presence 
equivalence  classes  to  be  established,  that  is, 
maintaining  the  same  level  of  measured  pres¬ 
ence,  whilst  trading  off  contributing  factors 
against  each  other.  However,  testing  such  a 
model's  empirical  validity  will  critically  depend  on 
sufficiently  reliable,  valid,  and  sensitive  meas¬ 
ures  of  presence  which  are  currently  being  re¬ 
searched  (for  an  overview  see  van  Baren  & 
IJsselsteijn,  2004;®),  but  can  not  yet  be  said  to 
exist  at  this  point  in  time. 

ESTABLISHING  SENSORIMOTOR  DEPEND¬ 
ENCIES  THROUGH  ACTIVE  EXPLORATION 

Perception  serves  the  individual's  need  to  con¬ 
trol  relevant  moment-to-moment  behaviour  or 
action  within  a  changing  environment.  The  de¬ 
velopment  of  visual  perception  of  object  shape 
and  environmental  layout  Is  strongly  dependent 
on  consistent  correlations  between  vision  and 
input  from  other  sensory  systems  (mainly  touch 
and  kinesthetics)  through  active  exploratory 
behaviour  of  the  environment,  establishing  a 
stable  yet  flexible  multisensory  representation 
of  space. 

In  a  classic  study,  Held^°  convincingly  demon¬ 
strated  the  relation  between  locomotor  experi¬ 
ence  and  the  understanding  of  spatial  relations. 
He  designed  an  experiment  using  a  ‘kitten  car¬ 
ousel’  in  which  kittens,  who  were  raised  from 
birth  in  total  darkness,  were  placed  in  pairs. 
One  kitten  pulled  the  carousel,  thus  using  its 
movement  to  determine  what  it  saw.  The  other 
kitten  was  placed  in  a  basket  at  the  other  end  of 
the  carousel,  which  was  controlled  by  the  first 
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kitten.  In  this  way,  the  second  kitten  received 
identical  visual  stimulation  to  the  first  kitten,  yet 
without  the  ability  to  control  the  visual  input. 
Between  these  sessions  of  visual  exposure,  the 
kittens  were  returned  to  the  dark.  After  42  days 
of  3  hour  sessions,  the  effects  of  active  versus 
passive  movement  became  strikingly  apparent 
when  the  kittens  were  tested  on  a  visual  cliff 
surface.  The  kittens  who  had  been  active  in  the 
carousel  shied  away  from  the  cliff  and  appropri¬ 
ately  stretched  out  their  legs  to  land  on  the 
lower  surface.  The  passive  kittens  showed 
none  of  that  behaviour,  thereby  providing  sup¬ 
port  for  the  view  that  the  development  of  per¬ 
ception  is  action-dependent.  Observations  in 
humans  have  led  to  similar  conclusions.  For 
instance,  Verkuyl^^  indicated  that  so-called  Sof¬ 
ten  on  children,  who  do  not  possess  upper  ex¬ 
tremities  due  to  the  use  of  a  sleeping  drug  in 
the  early  stages  of  the  mother’s  pregnancy,  had 
severe  problems  in  3D-perception. 

Studies  of  adaptation  to  prismatic  displace¬ 
ments  provide  further  support  for  the  impor¬ 
tance  of  establishing  reliable  sensorimotor  cor¬ 
relation  maps  through  actively  negotiating  the 
physical  environment.  Held  &  Hein^^  studied 
prismatic  displacements  in  humans  under  three 
conditions:  active  arm  movement,  passive  arm 
movement,  and  no  arm  movement.  In  the  active 
arm  movement  condition  the  subject  swung  her 
arm  back  and  forth  in  the  frontal  plane.  In  the 
passive  condition  it  was  transported  in  the 
same  manner  by  means  of  a  moving  cradle  to 
which  it  was  strapped.  Results,  as  measured  in 
terms  of  visual-motor  negative  after-effects, 
showed  that  adaptation  was  only  produced  in 
the  active  movement  condition  and  not  in  the 
passive  or  no-movement  conditions.  Another 
classic  experiment  on  visual  displacement^^ 
used  lenses  that  turn  the  world  upside-down. 
The  study  showed  that  full  adaptation  to  this 
situation  (i.e.,  seeing  the  world  right  side  up 
again)  occurred  after  a  few  days  only  when 
subjects  were  allowed  to  actively  explore  a 
complex  world.  When  a  subject  was  simply 
pushed  around  in  a  wheelchair,  he  did  not  show 
this  perceptual  adaptation  to  the  lenses. 

Similarly,  telemanipulation  experiments  using 
the  Delft  Virtual  Window  System^"*  demon¬ 
strated  a  significant  perceptual  advantage  of 
active  observers,  whose  head  movements  con¬ 


trolled  the  movements  of  a  remote  camera 
(generating  movement  parallax),  over  passive 
observers,  who  received  identical  visual  input 
(i.e.  motion  parallax),  yet  without  the  ability  to 
actively  change  the  viewpoint.  This  is  in  line 
with  results  found  in  relation  to  virtual  environ¬ 
ments,  where  Welch  et  al.’®  showed  that  partici¬ 
pants  who  had  active  control  over  a  simulated 
environment  indicated  higher  levels  of  presence 
than  participants  who  were  passively  exposed 
to  the  same  environment. 

BRAIN  PLASTICITY  AND  OUR  NEGOTIABLE 
BODY-IMAGE 

The  fact  that  technology  can  start  working  as  a 
transparent  extension  of  our  own  bodies  is  criti¬ 
cally  dependent  on  the  highly  plastic  nature  of 
our  brain,  which  is  continuously  able  and  prone 
to  adapt  to  altered  sensorimotor  contingencies, 
as  the  studies  on  adaptation  to  prismatic  dis¬ 
placement  and  tele-systems  already  illustrated. 
This  fact  finds  its  basis  in  the  significant  evolu¬ 
tionary  benefit  of  having  a  negotiable  body  im¬ 
age  to  accommodate  lifetime  development  and 
change,  which  requires  a  continuous  re¬ 
mapping  of  bodily  boundaries.  Although  body- 
image  adaptations  across  the  lifespan  can  af¬ 
ford  to  take  their  time,  it  is  the  relative  speed  of 
these  sensorimotor  adaptations  that  enables  us 
to  experience  man-made  technology  as,  quite 
literally,  part  of  ourselves  -  be  they  a  blind  per¬ 
son's  cane  or  an  advanced  telerobotic  arm. 

Further  evidence  of  the  high-speed  plasticity  of 
the  body  image  is  provided  by  the  amazing  ad¬ 
aptation  processes  that  may  occur  in  the  body- 
image  of  people  with  an  amputated  limb. 
Ramachandran,  Rogers-Ramachandran  and 
Cobb^®  had  (amputated)  phantom  limb  patients 
view  their  intact  arm  in  a  mirror,  such  that  their 
amputated  arm  appeared  to  be  resurrected. 
Several  subjects  reported  feeling  their  phantom 
limb  touched  when  they  viewed  the  mirror  im¬ 
age  of  their  intact  arm  being  touched  -  see  also 
Ramachandran  &  Blakeslee.^^ 

Botvinick  and  Cohen^®  provided  a  first  descrip¬ 
tion  of  the  ‘rubber  hand  illusion’.  This  crossmo- 
dal  perceptual  illusion  occurred  when  partici¬ 
pants’  left  hand  was  placed  out  of  view  and  a 
life-size  rubber  model  was  placed  in  front  of 
them.  Subsequently,  both  the  rubber  hand  and 
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participants’  left  hand  were  gently  stroked  by 
two  small  paintbrushes,  synchronizing  timing  as 
closely  as  possible.  In  line  with  the 
Ramachandran  et  al.^®  results,  subjects  re¬ 
ported  feeling  a  sense  of  ownership  of  the  rub¬ 
ber  hand,  as  if  it  was  actually  their  own. 
Ramachandran,  Hirstein  and  Rogers- 
Ramachandran^®  further  extended  this  work, 
showing  that  this  illusion  also  works  when 
physical  similarity  is  absent.  Instead  of  using  a 
rubber  hand,  they  simply  stroked  and  tapped 
the  tabletop  for  about  a  minute.  In  this  case, 
subjects  also  reported  sensations  arising  from 
the  tabletop.  They  argue  that  the  illusion  mainly 
arises  “from  the  ‘Bayesian  logic’  of  all  percep¬ 
tion;  the  brain’s  remarkable  ability  to  detect  sta¬ 
tistical  correlations  in  sensory  inputs  in  con¬ 
structing  useful  perceptual  representations  of 
the  world  -  including  one’s  body.’’(p.  1500).  In 
a  further  study,  Armel  and  Ramachandran®® 
showed  that  when  the  physical  integrity  of  the 
rubber  hand  was  threatened  (bending  a  finger 
backwards  to  seem  painful),  a  clear  skin  con¬ 
ductance  response  was  generated,  and  that  the 
illusion  could  even  be  projected  to  anatomically 
impossible  locations.  Using  the  rubber  hand 
illusion,  Ehrsson,  Spence,  and  Passingham®^ 
have  recently  studied  the  neuronal  counterparts 
of  the  feeling  of  body  ownership.  On  fMRI 
measures  of  brain  activity,  they  found  that  ar¬ 
eas  in  the  premotor  cortex  reflected  the  feeling 
of  ownership  of  the  rubber  hand,  indicating  that 
the  multisensory  integration  in  this  region  pro¬ 
vides  a  mechanism  for  bodily  self-attribution. 

The  results  on  perceptual  adaptation  described 
in  the  previous  paragraph,  as  well  as  the  rubber 
hand  illusion,  both  seem  to  suggest  that  what 
matters  most  is  a  closely  and  continuously  cor¬ 
related  loop  of  sensory  input,  neural  com¬ 
mands,  and  motor  action.  Thus,  modality- 
specific  feature  maps  appear  to  project  onto 
one  another  through  re-entrant  connections, 
which  allows  disjunctive  feature  characteristics 
(e.g.,  visual  and  haptic  properties  of  a  stimulus) 
to  be  connected  in  the  responses  of  higher- 
order  networks.  Sensorimotor  correlations  will 
initially  be  driven  by  the  temporally  ongoing  par¬ 
allel  signalling  between  primary  cortical  areas 
receiving  the  sense  data  associated  with  stimu¬ 
lus  objects  at  a  given  time  and  place.  Next,  sta¬ 
ble  feature  correlations  establish  reciprocal  con¬ 
nections  between  previously  disjunct  feature 


maps,  thus  allowing  for  higher  order  perception 
and  categorisation  of  objects  and  environ¬ 
ments.®®  The  ensuing  body  image  is  most  use¬ 
fully  conceptualised  as  a  temporary  construct 
which  is  remarkably  flexible,  and  may  include 
non-biological  artefacts  as  an  integral  part  of  it. 

FROM  FAR  TO  NEAR:  REMAPPING  SPACE 
IN  TELEPRESENCE 

In  general,  the  space  that  surrounds  the  user 
can  be  meaningfully  segmented  into  a  number  of 
ranges,  usually  three  or  four,  based  on  principles 
of  human  perception  and  action.  Several  models 
have  been  proposed,®®'®^  all  of  which  distinguish 
between  a  peripersonal  space  (the  immediate 
behavioural  space  surrounding  the  person)  and 
a  far  or  extrapersonal  space.  Referring  to  haptic 
space,  the  peripersonal  space  corresponds  to 
what  Lederman,  Klatzky,  Collins  &  Wardell  refer 
to  as  the  manipulatory  space,  i.e.,  within  hand’s 
reach,  whereas  the  extrapersonal  realm  would 
be  regarded  as  ambulatory  space,  requiring  ex¬ 
ploration  by  movements  of  the  body,  or  through 
the  use  of  a  tool  which  extends  the  bodily  reach. 
Animal  and  human  brain  studies  have  confirmed 
this  distinction  between  peripersonal  and  ex¬ 
trapersonal  space,  showing  that  space  is  not 
homogeneously  represented  in  the  brain.®®  ®® 

Telepresence  technologies  can  be  viewed  as 
attempts  to  overcome  the  boundaries  of  spatial 
segmentation.  Their  success  in  doing  so  is  evi¬ 
denced  by  a  clinical  case,  described  by  Berti  and 
Frassinetti,®^  where  a  patient  (P.P.),  after  a  right 
hemisphere  stroke,  showed  a  dissociation  be¬ 
tween  near  and  far  spaces  in  the  manifestation 
of  severe  visuo-spatial  neglect.  Using  a  line  bi¬ 
section  task,  the  neglect  was  apparent  in  near 
space,  but  not  in  far  space  when  bisection  in  the 
far  space  was  performed  with  a  projection  light 
pen.  However,  neglect  appeared  in  the  far  space 
as  well  when  the  line  bisection  task  was  per¬ 
formed  with  a  stick  (used  by  the  patient  to  reach 
the  line)  and  it  was  as  severe  as  neglect  in  the 
near  space.  Thus,  this  study  provides  evidence 
that  an  artificial  extension  of  a  person’s  body  (the 
stick)  causes  a  remapping  of  far  space  as  near 
space  -  essentially  telepresence. 

CONCLUSION 

The  same  sensorimotor  and  brain  systems  re¬ 
sponsible  for  our  sense  of  bodily  boundaries  and 
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our  sense  of  spatial  location  are  also  remarkably 
adaptable  to  include  non-biological  elements 
within  the  perceptual-motor  loop,  provided  reli¬ 
able,  real-time  sensorimotor  correlations  can  be 
established.  When  we  interact  with  virtual  or  re¬ 
mote  environments  using  intuitive  interaction 
devices,  isomorphic  to  our  sensorimotor  abilities, 
the  real-time,  reliable  and  persistent  chain  of 
user  action  and  system  feedback  will  effectively 
integrate  the  technology  as  a  phenomenal  exten¬ 
sion  of  the  self.  This  fluid  integration  of  technol¬ 
ogy  into  the  perceptual-motor  loop  eventually 
may  blur  the  boundary  between  our  ‘unmediated’ 
self  and  the  ‘mediating’  technology. 

Naive  definitions  of  ‘self  as  everything  contained 
within  our  bodily  boundaries,  and  ‘non-self  as 
the  world  outside  our  own  bodies  become  much 
less  obvious  when  we  regard  the  intimate  de¬ 
pendencies  and  co-adaptation  we  can  experi¬ 
ence  when  technology  starts  working  as  a  trans¬ 
parent  extension  of  our  own  bodies  and  minds. 
As  cognitive  scientist  Andy  Clark  convincingly 
argues  in  his  wonderful  book  ‘Natural  Born  Cy¬ 
borgs’  ,  what  ‘I’  am  is  not  defined  by  the  outer 
limits  of  the  ‘biological  skin-bag’.  He  states:  “For 
our  sense  of  self,  of  what  we  know  and  of  who 
and  what  we  are,  is  surprisingly  plastic  and  re¬ 
flects  not  some  rigid  preset  biological  boundary 
so  much  as  our  ongoing  experience  of  thinking, 
reasoning,  and  acting  within  whatever  potent 
web  of  technology  and  cognitive  scaffolding  we 
happen  currently  to  inhabit”  (p.45).^®  Thus  we 
learn  that  our  relationship  with  technology  is  a 
two-way  adaptive  process  -  we  adapt  the  tech¬ 
nologies  to  fit  our  needs  and  abilities  (a  process 
also  known  as  user-centred  design),  but  at  the 
same  time,  our  brain  adapts  itself  to  the  technol¬ 
ogy,  so  that  the  technology  becomes  part  of  our 
extended  self  -  the  biological  self  and  all  non- 
biological  tools  and  toys  we  employ  to  enhance 
our  performance  and  pleasure. 
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Abstract:  Memory  impairments  are  one  of  the  most  disabiing  consequences  of  brain  injury.  They 
can  be  problematical  to  assess  and  rehabilitate,  as  conventional  "paper  and  pencil"  neuropsychologi¬ 
cal  assessments  and  rehabilitation  strategies  are  often  criticised  as  lacking  in  relevance  to  the  use  of 
memory  in  the  real  world  i.e.,  "ecological  validity."  Conventional  neuropsychological  memory  assess¬ 
ments  and  rehabilitation  strategies  also  often  fail  to  cover  important  aspects  of  ‘everyday  memory,  ’ 
such  as  prospective  memory  (remembering  to  perform  actions  in  the  future)  and  incidental  memory 
(the  encoding  and  recall  of  information  without  any  instructions  from  a  third  party).  Unfortunately,  it 
can  be  very  difficult  to  achieve  the  ecological  validity  necessary  to  assess  everyday  memory  without 
compromising  experimental  control.  Similarly,  it  is  often  difficult  to  devise  rehabilitation  strategies  that 
involve  real  life  scenarios  owing  to  practical  constraints.  Virtual  reality  (VR)  provides  an  elegant  solu¬ 
tion  to  these  problems  and  is,  therefore,  an  ideal  tool  with  which  to  study  everyday  memory.  This  pa¬ 
per  briefly  summarises  the  current  state  of  research  on  the  applications  of  VR  to  everyday  memory 
assessment  and  rehabilitation,  with  reference  to  some  seminal  VR  work  within  the  domains  of  spa¬ 
tial,  incidental  and  prospective  memory.  Some  directions  for  future  research  are  suggested,  and  is¬ 
sues  associated  with  turning  empirical  findings  into  real  benefits  for  patients  considered. 


INTRODUCTION 

Over  the  last  decade,  the  potential  of  virtual 
reality  (VR)  has,  at  last,  begun  to  be  realised.  In 
the  early  to  mid-nineties,  the  aspirations  for  the 
technology  far  exceeded  its  capacity  to  deliver 
on  its  potentiall.^  However,  massive  advances 
in  the  sophistication  of  the  hardware  that  under¬ 
pins  VR  and  significant  reductions  in  the  cost  of 
such  hardware  have  taken  VR  “out  of  the  realm 
of  expensive  toy  and  into  that  of  functional  tech¬ 
nology”  (p3)." 

The  incorporation  of  VR  into  neuropsychologi¬ 
cal  assessment  and  rehabilitation  has  possibly 
attracted  the  greatest  expansion  of  interest 
from  researchers  in  recent  times.  Many  authors 
have  argued  that  the  assets  of  VR  are  espe¬ 
cially  well  suited  to  this  field. VR  has  been 
applied  to  a  number  of  domains  of  cognitive 
assessment  and  rehabilitation,  such  as:  execu¬ 
tive  functioning;®  attention  and  spatial  trans¬ 
formation.®  However,  as  Brooks  et  al.®  point 
out,  the  application  of  VR  to  memory  impair¬ 
ments  has  been  of  particular  interest,  as  mem¬ 
ory  disorders  are  one  of  the  most  disabling  con¬ 
sequences  of  brain  damage.^®  Therefore,  es¬ 


tablishing  whether  a  patient  has  memory  im¬ 
pairments  is  a  crucial  aspect  of  cognitive  as¬ 
sessment  and  addressing  any  such  impairment, 
a  crucial  aspect  of  cognitive  rehabilitation. 

1.1.  Reconciling  ecological  validity  and  reliabil¬ 
ity:  VR  and  everyday  memory. 

There  are  numerous  "paper  and  pencil"  neuro¬ 
psychological  tests  of  memory,  e.g.  the 
Wechsler  Memory  Scale  -  Revised,”  the  Adult 
Memory  and  Information  Processing  test,^®  the 
Recognition  Memory  Test,^®  and  the  Doors  and 
People  Test.”  However,  conventional  neuro¬ 
psychological  tests  have  been  heavily  criticized 
as  lacking  "ecological  validity"  or,  put  another 
way,  relevance  to  memory  under  conditions 
other  than  the  laboratory/rehabilitation  ward.^® 
Indeed,  it  was  the  poor  correlations  between 
laboratory  tests  and  ‘everyday  memory’  assess¬ 
ments  found  by  Sunderland,  Harris  and 
Baddeley^®  that  prompted  the  development  of 
the  Rivermead  Behavioural  Memory  Test 
(RBMT),  which  is  designed  to  assess  everyday 
memory.  However,  it  has  been  argued  that  the 
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cost  of  introducing  ecological  validity  into  con¬ 
ventional  neuropsychological  assessment  has 
been  a  loss  of  strict  control  over  the  test  situa¬ 
tion.’’^ 


The  problem  of  balancing  the  demands  of  eco¬ 
logical  validity  with  sufficiently  rigorous  control  is 
also  apparent  in  rehabilitation.  Doubts  have 
been  raised  as  to  whether  rehabilitation  strate¬ 
gies  that  lack  ecological  validity  will  transfer  to 
the  real  world^®  and  whether  training  behaviours 
in  the  absence  of  any  change  in  the  environ¬ 
ment,  or  task  demands,  fosters  sufficient  intel¬ 
lectual  flexibility  and  creativity  to  be  useful  in 
everyday  life.’’®  Therefore,  of  all  the  assets  of 
VR  for  assessment  and  rehabilitation  (see 
Rizzo,  Schultheis,  Kerns  and  Mateer,^  for  a 
comprehensive  review)  it  is  perhaps  VR's  ca¬ 
pacity  to  generate  environments  that  are  satisfy¬ 
ing  from  an  ecological  standpoint,  without  com¬ 
promising  on  experimenter/therapist  control, 
that  is  most  exciting  for  memory  assessment 
and  rehabilitation  research.  This  capacity  brings 
with  it  exciting  opportunities  to  study  everyday 
memory  phenomena.  Indeed,  applications  of 
VR  to  memory  research  to  date  have  predomi¬ 
nantly  focused  on  three  domains  of  everyday 
memory,  which  have  proved  particularly  prob¬ 
lematical  for  conventional  assessment  and  re¬ 
habilitation:  i.e.,  spatial  memory,  incidental 
memory  and  prospective  memory.  The  following 
provides  an  illustration  of  how  the  assets  of  VR 
have  contributed  to  our  understanding  of  the 
above  domains  of  everyday  memory,  with  refer¬ 
ence  to  some  significant  VR  research. 

VR  AND  SPATIAL  MEMORY 

A  good  deal  of  the  developing  literature  on  eve¬ 
ryday  memory  assessment  and  rehabilitation  in 
VR  to  date  has  been  directed  at  spatial  mem¬ 
ory,  arguably  for  two  reasons.  Firstly,  as  Brooks 
et  al.®  state,  adequate  spatial  memory  is  a  pre¬ 
requisite  for  independent  living.  Secondly,  VR 
spatial  memory  tasks  are  particularly  amenable 
to  being  orientated  to  utilise  users’  procedural 
memory  for  motor  tasks  e.g.,  simulated  naviga¬ 
tion  between  waypoints.  This  has  significant 
implications  for  the  reorganisation  of  memory 
function  in  individuals  who  have  suffered  brain 
damage,  which  will  be  elaborated  on  below. 

One  of  the  primary  assets  of  VR  in  everyday 
memory  assessment  is  the  capacity  to  perform 


experiments  that,  though  methodologically  de¬ 
sirable,  would  not  be  practical  in  the  real  world. 
This  is  particularly  true  for  research  into  spatial 
memory.  A  good  example  would  be  the  VR  ad¬ 
aptation  of  the  Morris  Water  Task,  an  experi¬ 
mental  paradigm  that  had  proved  highly  infor¬ 
mative  in  investigating  spatial  ability  in  rodents, 
to  research  into  human  spatial  ability.  The  task 
involves  finding  a  platform,  which  is  submerged 
in  water,  purely  by  using  memory  of  its  position 
relative  to  external  environmental  cues.  A  desk¬ 
top  VR  variant  of  this  paradigm  has  proved  use¬ 
ful  in  assessing  spatial  learning  decrements  in 
TBl  patients,  which  corresponded  to  self-  re¬ 
ported  frequency  of  place  learning  problems  in 
everyday  life.^° 

Using  the  control  and  ecological  validity  af¬ 
forded  by  VR  together  with  fMRI  generates 
unique  and  exciting  insights  into  the  neurologi¬ 
cal  correlates  of  everyday  memory,  which  would 
not  be  possible  in  the  real  world  owing  to  port¬ 
ability  issues  inherent  in  fMRI  equipment.  For 
example,  Morris  et  al.^^  and  Paslow  et  al.^^  used 
a  PC-based  virtual  reality  set  up  in  conjunction 
with  fMRI  in  order  to  examine  the  brain  corre¬ 
lates  of  egocentric  memory  (spatial  knowledge 
relative  to  the  observer)  and  allocentric  memory 
(spatial  knowledge  relative  to  cues  independent 
of  the  observer)  in  patients  with  anoxic  hippo¬ 
campal  damage  and  normal  controls.  The  re¬ 
sults  from  these  studies  provided  an  in  vivo 
demonstration  of  how  anoxic  brain  damage  can 
affect  hippocampal  function. 

In  addition  to  overcoming  practical  difficulties 
with  implementing  a  desired  methodology  or 
permitting  the  incorporation  of  relatively  inac¬ 
cessible  medical  technology,  a  brief  look  at  the 
use  of  VR  in  the  rehabilitation  of  spatial  memory 
reveals  that  it  can  also  directly  facilitate  the  re¬ 
organisation  of  memory  ability.  Research  has 
shown  that  procedural  memory  for  motor  tasks 
can  often  remain  relatively  intact,  even  in  indi¬ 
viduals  who  exhibit  profound  impairments  in 
other  aspects  of  memory-  as  in  the  widely  re¬ 
ported  case  of  HM.^’^  Research  has  also  indi¬ 
cated  that  one  can  utilize  intact  procedural 
memory  to  aid  or  replace  damaged  memory 
systems.^'*'^®  Unfortunately,  conducting  spatial 
navigation  tasks  involving  more  than  one  room, 
or  in  an  environment  relevant  to  the  patient’s 
everyday  life,  can  be  highly  problematical  in  the 
real  world.  This  problem  is  easily  rectified  within 
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VR,  the  question  was:  is  a  simulation  of  a  motor 
activity  sufficient  to  tap  into  spared  procedural 
memory  in  an  amnesic  patient?  Brooks  et  al.^® 
used  a  desktop  VR  system  to  train  an  amnesic 
patient  (MT)  to  independently  navigate  between 
locations  within  her  hospital’s  rehabilitation  unit. 
When  the  training  was  extended  to  encompass 
real  world  and  VR  based  training  of  routes 
(matched  for  complexity)  within  the  ward,  it  tran¬ 
spired  that  MT  learned  the  route  practised  in 
VR,  but  not  the  route  practised  in  the  real  world. 
A  number  of  features  of  the  VR  rehabilitation 
may  have  been  responsible  for  this  outcome. 
For  example,  MT  could  perform  the  route  very 
quickly  in  VR  resulting  in  more  repetitions  of  the 
VR  route  than  the  route  practised  in  the  real 
unit.  The  rehabilitation  strategy  of  backwards 
training  was  easier  to  implement  in  VR  than  in 
the  real  world.  Finally,  extraneous  distractions 
were  easily  controlled  in  VR.  A  follow  up  study 
reported  in  Rose  et  al.^®  provides  further  sup¬ 
port  for  the  use  of  VR  for  route  training  in  amne¬ 
sic  patients. 

Researchers  have  also  shown  interest  in  VR  as  a 
means  to  encourage  the  development  of  spatial 
memory  In  children  whose  physical  disabilities 
restrict  their  mobility.  For  example,  Stanton  et 
al;^®  Stanton  et  aP°  have  conducted  experiments 
featuring  a  VR  model  of  a  real  school,  which 
physically  disabled  children  were  asked  to  inde¬ 
pendently  explore  with  a  view  to  determining 
whether  VR-based  spatial  memory  training  trans¬ 
lates  into  improvements  in  performance  in  the 
real  environment  on  which  the  virtual  environ¬ 
ment  was  modelled.  Subsequent  testing  of  the 
children  on  their  spatial  knowledge  of  the  real 
school  indicated  that  they  performed  better  at 
pointing  tasks  and  showed  better  route  knowl¬ 
edge  than  undergraduate  students  who  had  not 
used  the  virtual  environment.  Similarly,  Wilson  et 
al.^^  conducted  an  experiment,  which  featured  a 
virtual  reality  reproduction  of  two  floors  of  the 
Astley  Clarke  building  at  the  University  of  Leices¬ 
ter,  UK.  The  children  were  asked  to  activate  fire 
equipment  and  open  a  fire  door  to  escape  from 
the  building  in  VR.  Subsequent  assessment  in¬ 
volved  asking  the  children  to  identify  items  of  fire 
equipment  that  would  not  have  been  visible  from 
their  vantage  points  and  the  identification  of  the 
shortest  route  to  the  fire  exit.  The  children  out¬ 
performed  a  control  group  of  undergraduate  stu¬ 
dents  who  had  not  been  exposed  to  the  virtual 
environment.  These  studies  highlight  another 


advantage  of  using  VR  in  everyday  memory  as¬ 
sessment;  that  the  spatial  memory  of  physically 
disabled  participants  can  be  assessed  using  VR 
in  some  situations  where  other  forms  of  assess¬ 
ment  might  not  be  possible. 

VR  AND  INCIDENTAL  MEMORY 

Incidental  memory  is,  perhaps,  the  type  of  mem¬ 
ory  most  relevant  to  everyday  life  as,  unlike  in 
conventional  memory  tests,  often  in  day-to-day 
life  individuals  are  not  made  explicitly  aware  that 
they  will  be  required  to  remember  a  particular 
stimulus  prior  to  having  reason  to  recall  it.  This 
scenario  is  difficult  to  achieve  in  a  laboratory,  as 
an  everyday  environment  is  usually  an  integral 
part  of  the  task.  However,  VR  is  ideally  suited  to 
conducting  incidental  memory  assessments,  as  it 
can  easily  mimic  real-life  environments  and 
tasks,  and  participants  can  then  be  incidentally 
tested  on  items  encountered  during  their  per¬ 
formance  of  the  tasks.  Despite  this,  the  use  of 
VR  in  incidental  memory  studies  is  still  rare. 

Andrews  et  conducted  a  pioneering  study  of 
incidental  memory  for  objects  in  VR.  Participants 
examined  a  series  of  objects  in  one  of  five  condi¬ 
tions:  during  interaction  with  a  four-room  virtual 
environment:  in  four  static  displays  without  any 
context:  in  the  same  four  static  displays,  but 
where  the  participants  were  required  to  move  a 
cursor  over  each  object  in  turn;  in  four  static  pic¬ 
tures  of  virtual  rooms:  and  using  the  same  four 
pictures,  but  where  the  participants  were  re¬ 
quired  to  move  a  cursor  over  each  of  the  four 
objects  in  turn.  Subsequent  object  recognition 
tests  revealed  that  the  poorest  recognition  per¬ 
formance  occurred  in  the  condition  where  partici¬ 
pants  encountered  the  objects  in  the  virtual  envi¬ 
ronment.  Andrews  et  al.  concluded  that  partici¬ 
pants  were  being  distracted  by  their  interaction 
with  the  virtual  environment  and  that  incidental 
memory  is  particularly  susceptible  to  distraction. 
However,  they  cautioned  that  the  virtual  interac¬ 
tion  condition  was  most  representative  of  partici¬ 
pants’  real-world  memory  ability. 

Rose  et  al.^^  conducted  an  experiment  to  investi¬ 
gate  incidental  spatial  and  incidental  object  rec¬ 
ognition  memory  in  vascular  brain  injury  patients 
and  controls.  This  experiment  involved  the  pres¬ 
entation  of  a  four  room  virtual  bungalow  that  par¬ 
ticipants  were  asked  to  search,  ostensibly  for  a 
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toy  car.  A  yoked-control  design  was  used,  in 
which  half  of  each  group  of  participants  were 
assigned  to  an  ‘active’  condition,  which  entailed 
controlling  movement  through  the  bungalow  via  a 
joystick.  The  other  half  were  assigned  to  a 
‘passive’  condition,  which  entailed  watching  the 
active  participant's  progress  on  a  second  monitor 
in  an  adjoining  cubicle.  Participants  were  as¬ 
sessed  via  a  spatial  layout  recognition  test,  in 
which  they  were  required  to  assemble  2-D  maps 
of  the  virtual  environment  by  choosing  room 
shapes  and  the  positions  of  entry  and  exit  door¬ 
ways  from  an  array  of  possibilities.  Active  experi¬ 
ence  of  the  virtual  environment  was  found  to  im¬ 
prove  incidental  spatial  recognition  memory  of 
the  layout  of  the  four-room  building  for  both  pa¬ 
tients  and  controls.  Conversely,  passive  observa¬ 
tion  was  found  to  improve  controls’  performance 
in  the  object  recognition  test,  but  not  patients' 
performance.  Using  the  same  procedure  Pugnetti 
et  al.^  and  Rose  et  al.^®  replicated  these  findings 
with  multiple  sclerosis  patients  and  people  with 
learning  disabilities,  respectively.  Pugnetti  et  al.^ 
also  found  that  the  above  finding  with  MS  pa¬ 
tients  could  not  be  accounted  for  via  standard 
pencil  &  paper  memory  assessment  tools.  Thus, 
there  is  evidence  that,  in  addition  to  providing  an 
experimental  tool  more  amenable  to  the  study  of 
incidental  memory  per  se,  VR  may  also  provide  a 
more  sensitive  test  of  incidental  memory,  particu¬ 
larly  in  impaired  populations. 

VR  AND  PROSPECTIVE  MEMORY 

Impaired  prospective  memory  is  a  particularly 
disabling  form  of  memory  impairment,  and  poten¬ 
tially  the  most  hazardous,  as  often  in  everyday 
life  actions  that  one  needs  to  remember  to  per¬ 
form  at  some  point  in  the  future  are  inherently 
important,  e.g.  remembering  to  attend  a  meeting, 
or  turn  off  the  cooker.  Consequently,  the  need  for 
a  comprehensive  assessment  of  prospective 
memory  as  part  of  a  cognitive  rehabilitation  pro¬ 
gramme  is  self-evident.  However,  performing  an 
ecologically  valid,  yet  rigorously  controlled  as¬ 
sessment  of  prospective  memory  ability  in  a  re¬ 
habilitation  setting  is  problematical.  Kvavilashvili 
and  Ellis^  classified  prospective  memory  accord¬ 
ing  to  the  type  of  cue  the  particular  task  to  be 
remembered  entailed,  i.e.  event-based  tasks  (an 
action  is  required  to  be  performed  in  response  to 
a  cue):  time-based  tasks  (an  action  is  required  at 
a  previously  specified  future  time);  or  activity- 


based  tasks  (an  action  is  required  before  or  after 
performing  an  activity). 

Based  on  this  classification.  Brooks  et  al.®  de¬ 
vised  and  conducted  a  desktop  VR  prospective 
memory  test  on  patients  from  a  stroke  rehabilita¬ 
tion  unit  and  age-matched  controls.  The  virtual 
environment  used  in  the  test  was  a  four-room 
bungalow.  The  test  involved  participants  perform¬ 
ing  a  furniture  removal  task,  in  which  they  were 
required  to  instruct  the  experimenter  to  move 
furniture  and  items  from  the  virtual  bungalow  to 
the  appropriate  room  (out  of  a  possible  eight)  of 
a  new  house,  one  piece  at  a  time.  This  consti¬ 
tuted  the  background  task.  Before  performing 
this  task,  they  were  asked  to  remember  to  in¬ 
struct  the  experimenter  to  put  "Fragile"  notices  on 
five  glass  items  (the  event-based  prospective 
memory  task);  to  keep  the  kitchen  door  closed  to 
keep  the  cat  in  (the  activity-based  prospective 
memory  task);  and  to  return  to  a  clock  in  the  hall 
at  five-minute  intervals  and  press  a  button,  sup¬ 
posedly  to  let  removal  men  in  (the  time-based 
prospective  memory  task).  It  transpired  that  14  of 
the  36  stroke  patients  and  2  of  the  22  control 
participants  were  unable  to  recall  all  three  of  the 
prospective  memory  task  instructions  immedi¬ 
ately  after  they  had  finished  the  furniture  removal 
task,  even  though  they  had  been  able  to  recall 
them  immediately  prior  to  beginning  the  task.  A 
comparison  of  the  results  of  the  remaining  stroke 
patients  and  controls,  who  were  able  to  recall  all 
three  prospective  memory  tasks,  indicated  that 
the  stroke  patients  were  severely  impaired  rela¬ 
tive  to  controls  at  the  event  and  activity  based 
tasks,  but  only  marginally  impaired  at  the  time- 
based  task.  These  results  were  surprising  as, 
based  on  Craik’s^^  theory  of  age-related  decre¬ 
ments  in  memory,  which  proposes  that  self- 
initiated  processing  becomes  more  difficult  for 
the  elderly,  it  was  envisaged  that  impairments  in 
stoke  patients  would  be  more  apparent  in  time- 
based  tasks  (which  involve  self-initiated  retrieval) 
than  in  event  or  activity-based  tasks  (which  fea¬ 
ture  external  cues).  If,  as  these  results  of  this 
study  indicate,  self-initiated  retrieval  does  not 
become  much  more  difficult  for  stroke  patients, 
their  failure  to  remember  to  perform  tasks  in  the 
future  in  this  study  may  be  attributable  to  a  differ¬ 
ent  impairment,  perhaps  an  inability  to  multi-task. 

Brooks  et  al’s.®  exploratory  study  indicates  that 
VR  can  provide  a  more  comprehensive  and  con¬ 
trolled  assessment  of  prospective  memory  ability 
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than  would  be  possible  using  standard  memory 
assessment.  This  contributes  to  our  understand¬ 
ing  of  the  neuropsychological  consequences  of 
stroke.  It  also  enables  rehabilitation  to  be  more 
effectively  directed  towards  the  specific  impair¬ 
ments  of  individual  patients. 

DIRECTIONS  FOR  FUTURE  RESEARCH  AND 
ISSUES  ASSOCIATED  WITH  TURNING  VR’S 
POTENTIAL  INTO  BENEFITS  FOR  PATIENTS 

The  literature  on  VR  and  spatial  memory  pro¬ 
vides  a  glimpse  of  the  exciting  potential  of  using 
VR  in  conjunction  with  fMRI  in  order  to  examine 
the  neurological  correlates  of  different  forms  of 
everyday  memory.^  Future  studies  could  utilize 
this  procedure  to  elucidate  the  relationship  be¬ 
tween  the  area  of  the  brain  affected  by  stroke, 
traumatic  brain  injury  (TBI),  or  dementia  and  im¬ 
paired  everyday  memory  ability.  Quite  apart  from 
being  theoretically  significant,  such  data  would 
permit  therapists  to  more  accurately  predict  and 
rehabilitate  the  particular  memory  impairments 
that  a  patient  would  be  likely  to  experience  on 
the  basis  of  their  fMRI  scan. 

VR  has  demonstrated  utility  in  assisting  people 
with  memory  impairments  by  utilizing  intact  pro¬ 
cedural  memory^®  this  learning  has  translated  to 
improved  performance  in  the  real  world.  How¬ 
ever,  it  is  probably  true  to  say  that  spatial  mem¬ 
ory  tasks  are  particularly  amenable  to  being  bro¬ 
ken  down  into  a  series  of  steps  that  tap  into  pro¬ 
cedural  memory.®  The  task  of  researchers  and 
therapists  is  now  to  ascertain  whether  this  use  of 
VR  can  be  successfully  applied  to  other  everyday 
tasks,  and  whether  it  can  be  used  to  negate 
some  of  the  other  disorientation,  disorganization 
and  repetition  disabilities  which  result  from  differ¬ 
ent  forms  of  memory  impairment.  Researchers 
will  also  have  to  be  creative  in  providing  ways  of 
overcoming  the  lack  of  conscious  awareness  of 
learned  knowledge  common  in  patients  with  am¬ 
nesia  who  have  learned  new  motor  skills.^'* 
Brooks  et  al  also  identified  numerous  reasons  for 
the  greater  efficacy  of  VR  training  relative  to  real 
world  training,  e.g.,  less  distraction,  more  fre¬ 
quent  repetition,  greater  amenability  of  VR  train¬ 
ing  to  the  errorless  learning  paradigm  etc.  It 
would  be  useful  for  the  design  of  future  memory 
rehabilitation  strategies  in  VR  to  have  some  data 
on  the  relative  importance  of  each  of  these  as¬ 
sets  in  contributing  to  rehabilitation  outcomes. 


The  implications  of  active  vs.  passive  exploration 
of  virtual  environments  for  incidental  spatial  lay¬ 
out  and  object  recognition  are  interesting  and 
potentially  very  significant.®®  The  impact,  positive 
or  otherwise,  of  active  and  passive  exploration 
appears  to  be  contingent  on  the  stimuli  and  task 
demands.  Rose  et  al.  speculated  that  active  ex¬ 
ploration  promoted  the  activation  of  procedural 
memory  in  the  active  participants,  thereby  en¬ 
hancing  their  spatial  memory  of  the  layout  of  the 
virtual  environment  compared  to  the  passive  par¬ 
ticipants.  The  active  participants’  advantage  did 
not  extend  to  enhanced  recall  of  the  objects  in 
the  virtual  environment  because  active  partici¬ 
pants  did  not  interact  with  the  virtual  objects. 
However,  it  should  be  noted  that  the  degree  of 
interaction  afforded  by  VR  in  memory  assess¬ 
ment  and  rehabilitation  is  something  of  a  double- 
edged  sword.  Increased  interaction  can  have  a 
deleterious  effect  on  memory  assessment  and 
rehabilitation,  as  amnesic  patients  can  have  diffi¬ 
culty  explicitly  distinguishing  between  correct  and 
incorrect  responses/actions®®  and  freedom  to 
examine  a  virtual  environment  can  prove  as 
much  of  a  distraction  as  an  advantage.®®  ®®  Thus, 
experimenters/therapists  must  be  careful  to  en¬ 
sure  that  the  range  of  interaction  offered  by  a 
virtual  environment  does  not  detract  from  the 
purpose  of  the  exercise  by  providing  careful  cue¬ 
ing  or  adopting  error  free  learning  approaches, 
such  as  that  employed  by.®® 

VR  has  been  shown  to  provide  a  more  compre¬ 
hensive,  ecologically  valid,  and  controlled  as¬ 
sessment  of  prospective  memory  ability  than  is 
currently  possible  using  standard  memory  as¬ 
sessment  tests.  However,  some  questions  re¬ 
main  from  Brooks  et  al.®  Firstly,  it  is  not  entirely 
clear  whether  the  lack  of  impairment  in  the  time- 
based  prospective  memory  task  in  stroke  pa¬ 
tients  relative  to  controls  may  have  been  an  arti¬ 
fact  of  the  interval  between  each  retrieval  occa¬ 
sion  being  too  short.  Thus,  this  aspect  of  the  re¬ 
sults  may  have  been  due  to  the  time-based  task 
not  being  sufficiently  representative  of  an  every¬ 
day  time-based  prospective  memory  task,  as 
opposed  to  a  genuine  reflection  on  preserved 
prospective  memory  capacity.  Further  research, 
utilizing  different  time  intervals  in  the  time-based 
task,  is  planned  to  investigate  this  effect. 

Brooks  et  al.  also  revealed  that  motivation  is  im¬ 
portant  in  prospective  memory  performance,  es- 
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pecially  for  stroke  patients.  During  this  study, 
participants  were  also  given  two  real  world  based 
prospective  memory  tasks  i.e.  remember  to  ask 
the  experimenter  to  return  a  personal  belonging 
at  the  end  of  the  study,  and  ask  for  a  written  ex¬ 
planation  of  the  study.  Results  indicated  that  par¬ 
ticipants  were  significantly  more  likely  to  remem¬ 
ber  the  former  task,  probably  because  they  were 
more  motivated  to  ask  for  the  return  of  their  prop¬ 
erty  than  for  the  written  explanation.  If  rehabilita¬ 
tion  strategies  can  encourage  stroke  patients  to 
be  more  motivated  to  remember,  it  seems  logical 
and  reasonable  to  assume  that  their  prospective 
memory  abilities  would  improve.  VR  is  particu¬ 
larly  well  suited  to  improving  motivation  in  reha¬ 
bilitation  by,  for  example,  introducing  gaming 
elements  into  rehabilitation  strategies.®  Re¬ 
search  is  currently  underway  to  determine 
whether  manipulation  of  the  salience  of  the  differ¬ 
ent  types  of  prospective  memory  tasks  (e.g., 
time,  event  and  activity  based)  can  influence  pa¬ 
tients’  capacity  to  remember  to  perform  them. 
This  may  be  particularly  useful  for  the  patients 
(and  controls)  in  Brooks  et  al.  who  were  unable 
even  to  remember  what  the  prospective  memory 
tasks  were. 

CONCLUSION 

The  work  conducted  to  date  clearly  indicates 
that  VR  has  very  significant  potential  in  the  as¬ 
sessment  and  rehabilitation  of  everyday  mem¬ 
ory  in  individuals  that  have  suffered  brain  injury. 
Further  work  is  needed  to  establish  whether  VR 
can  be  used  in  the  reorganization  of  everyd^ 
memory  tasks  other  than  simple  navigation.* 
Research  is  also  needed  to  further  elucidate  the 
implications  of  active  vs.  passive  exploration  of 
virtual  environments  and  the  relative  contribu¬ 
tion  of  the  other  assets  of  VR  to  everyday  mem¬ 
ory  assessment  and  rehabilitation  scenarios. 
Additional  work  involving  VR  used  in  conjunc¬ 
tion  with  fMRI  is  clearly  warranted  to  provide 
new  and  exciting  insights  into  the  neurological 
correlates  of  everyday  memory  and  gain  a 
greater  understanding  of  how  damage  to  differ¬ 
ent  regions  of  the  brain  impacts  upon  everyday 
memory  performance. 
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Abstract;  Virtual  Reality  (VR)  is  an  advanced  human-computer  interface  that  allows  the  user  to  inter¬ 
act  with  and  become  immersed  in  a  computer-generated  environment  in  a  naturalistic  fashion. 

Different  clinical  data  suggest  that  VR  can  help  in  addressing  two  key  features  of  eating  disorders 
and  obesity  not  always  adequately  addressed  by  existing  approaches:  body  experience  disturbances 
and  self-efficacy. 

In  fact,  performance-based  methods  are  the  most  effective  in  producing  therapeutic  change  across 
behavioral,  cognitive,  and  affective  modalities.  The  experiential  approach  allowed  by  VR  helps  pa¬ 
tients  in  discovering  that  difficulties  can  be  defeated,  so  improving  their  cognitive  and  behavioral 
skills  for  coping  with  stressful  situations.  On  the  downside,  the  cost  of  this  technology  remains  rela¬ 
tively  high,  its  availability  is  still  limited  and  some  patients  (1-2%)  may  experience  simulation  sick¬ 
ness. 

The  paper  discusses  the  pros  and  cons  of  this  approach  presenting  the  results  coming  from  the  last 
controlled  clinical  trials. 


INTRODUCTION 

Different  new  technologies  have  been  intro¬ 
duced  over  the  last  few  years  that  are  increas¬ 
ingly  finding  application  in  health  care  delivery 
for  patients  with  eating  disorders  and  obesity. 
These  inciude  self-help  (supervised  and  unsu¬ 
pervised),  telemedicine,  teiephone  therapy,  e- 
mail,  Internet,  computer  software,  CD-ROMs, 
portable  computers,  and  virtual  reality  tech¬ 
niques.^  One  of  the  most  promising  is  virtual 
reality  (VR),  an  advanced  form  of  human- 
computer  interface  that  allows  the  user  to  inter¬ 
act  with  and  become  immersed  in  a  computer¬ 
generated  environment  in  a  naturalistic  fash¬ 
ion.^ 

In  fact,  therapists  are  using  VR  to  provide  a 
new  human-computer  interaction  paradigm  in 
which  users  are  active  participants  within  a 
computer-generated  three-dimensional  virtual 
world. ^  Using  VR  in  this  way,  the  patient  is 
more  likely  not  only  to  gain  an  awareness  of 
his/her  need  to  do  something  to  create  change 
but  also  to  experience  a  greater  sense  of  per¬ 
sonal  efficacy.'' 


This  feature  of  VR  has  been  extensively  used  in 
different  clinical  psychology  treatments  -  from 
phobias  to  sexual  disorders  -  and  it  is  expected 
to  increase  in  the  future.^®  In  particular,  an  area 
in  which  VR  may  offer  a  competitive  advantage 
is  the  treatment  of  eating  disorders  and  obesity. 

Different  clinical  data  suggest  that  VR  can  help 
in  addressing  two  key  features  of  eating  disor¬ 
ders  and  obesity  not  always  adequately  ad¬ 
dressed  by  existing  approaches:  body  experi¬ 
ence  disturbances  and  self-efficacy.^ 

The  paper  discusses  the  pros  and  cons  of  this 
approach  presenting  the  results  coming  from 
the  last  controlled  clinical  trials. 

RATIONALE  AND  CLINICAL  DATA 

Distorted  body  image,  negative  emotions  and 
lack  of  faith  in  the  therapy  are  typical  features 
of  these  disturbances  and  are  the  most  difficult 
characteristics  to  change.  One  innovative  ap¬ 
proach  to  their  treatment  is  to  enhance  tradi- 
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Hardware 

Graphic  Workstation  with  high-end  graphic  card 

Head  Mounted  Display  or  3D  shutter  glasses 

Tracking  System  (Head  and  Hands) 

VR  Gloves 

Software 

VR  environment 

Table  1;  VR  Components 


tional  cognitive-behavioral  therapy  (CBT)  with 
the  use  of  a  virtual  environment. 

A  first  approach  is  the  one  offered  by  the  Inte¬ 
grated  Experiential  Therapy  (lET).  Developed 
by  Giuseppe  Riva  and  his  group  inside  the 
VREPAR  and  VEPSY  Updated  European 
funded  projects  (http://www.cybertherapy.info) 
is  a  relatively  short-term,  patient  oriented  ap¬ 
proach  that  focuses  on  individual  discovery.^°'^^ 
lET  shares  with  CBT  the  use  of  a  combination 
of  cognitive  and  behavioral  procedures  to  help 
the  patient  identify  and  change  the  maintaining 
mechanisms.  However  it  is  different  for: 

•  Its  use  of  Virtual  Reality  (VR):  10  VR  ses¬ 
sions. 

•  Its  focus  on  the  negative  emotions  related  to 
the  body,  a  major  reason  patients  want  to  lose 
weight. 

•  Its  focus  on  supporting  the  empowerment 
process.  VR  has  the  right  features  to  support 
empowerment  process,  since  it  is  a  special, 
sheltered  setting  where  patients  can  start  to 
explore  and  act  without  feeling  threatened. 

For  the  virtual  reality  sessions,  the  Virtual  Real¬ 
ity  for  Eating  Disorders  Modification  -  VREDIM  - 
is  used.  VREDIM  is  an  enhanced  version  of  the 
original  Virtual  Reality  for  Body  Image  Modifica¬ 
tion  (VEBIM)  immersive  virtual  environment, 
previously  used  in  different  preliminary  studies 
on  non-clinical  subjects^^’^'* 

VREDIM  is  composed  by  14  virtual  environ¬ 
ments,  used  by  the  therapist  during  a  50-minute 
session  with  the  patient.^’  After  a  first  assess¬ 
ment  session,  the  next  9  sessions  are  used  to 
assess  and  modify: 

•  the  symptoms  of  anxiety  related  to  food  ex¬ 
posure.  This  is  done  by  integrating  different 


cognitive-behavioral  methods  (see  Table  1): 
Countering,  Alternative  Interpretation,  Label 
Shifting,  Deactivating  the  Illness  Belief  and 
Temptation  Exposure  with  Response  Pre¬ 
vention.'’^''® 

•  the  body  experience  of  the  subject.  To 
reach  this  goal,  the  virtual  environment  inte¬ 
grated  the  therapeutic  methods  used  by 
Butter  &  Cash^^  and  Wooley  &  Wooley.^®  In 
particular  in  VREDIM  the  virtual  environ¬ 
ment  is  used  in  the  same  way  as  guided 
imagery^®  is  used  by  the  cognitive  and  vis- 
ual/motorial  approach.  Moreover,  in  terms 
of  learning  theory,  repeated  and  prolonged 
exposure  with  the  conditioned  stimulus 
“seeing  one’s  own  body”  is  supposed  to 
induce  decreases  in  the  conditioned  nega¬ 
tive  reactivity  by  preventing  negative  rein¬ 
forcement,  e.g.,  avoidance.^® 

The  VR  sessions  (see  Figure  1)  approximate 
natural  settings,  providing  an  alternative  for  ex¬ 
posure  and  desensitization  exercises  as  well  as 
a  more  general  enhancement  to  therapy.  Spe¬ 
cifically,  VR  is  believed  to  increase  motivation 
by  allowing  individuals  to  virtually  witness 
changes  in  their  behavior  and  shape  and  reach 
their  own  conclusions  based  on  actual  experi¬ 
ence.  During  a  typical  VR  sessions,  patients  are 
asked  to  wear  a  head  mounted  VR  display  sys¬ 
tem.  An  approach  similar  to  guided  imagery  is 
used  to  lead  the  subject  through  various  zones 
over  the  course  of  ten  sessions.  Stimuli  that 
contribute  to  abnormal  eating  behaviors  are 
identified  and  associated  anxiety  and  body  ex¬ 
periences  are  targeted  for  modification. 

Subjects  are  also  asked  to  identify  figures  that 
most  closely  resemble  their  current  and  ideal 
body  sizes.  They  are  also  confronted  with  a 
photograph  of  their  actual  body. 
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This  approach  was  validated  through  different 
case  studies^^  and  trials.  In  the  first  one,  uncon¬ 
trolled,  three  groups  of  patients  were  used^^: 
patients  with  Binge  Eating  Disorders  (BED), 
patients  with  Eating  Disorders  Not  Otherwise 
Specified  (EDNOS),  and  obese  patients  with  a 
body  mass  index  higher  than  35.  All  patients 
participated  in  five  biweekly  sessions  of  the 
therapy.  All  the  groups  showed  improvements  in 
overall  body  satisfaction,  disordered  eating,  and 
related  social  behaviors,  although  these 
changes  were  less  noticeable  in  the  EDNOS 
group. 

More  recently,  the  approach  was  tested  in  dif¬ 
ferent  controlled  studies.  The  first  one  involved 
twenty  women  with  BED  who  were  seeking  resi¬ 
dential  treatment.”  The  sample  was  assigned 
randomly  to  lET  or  to  CBT  based  nutritional 
therapy.  Both  groups  were  prescribed  a  1,200- 
calorie  per  day  diet  and  minimal  physical  activ¬ 
ity.  Analyses  revealed  that  although  both  groups 
were  binge  free  at  1 -month  follow-up,  lET  was 
significantly  better  at  increasing  body  satisfac¬ 
tion.  In  addition,  lET  participants  were  more 


likely  to  report  increased  self-efficacy  and  moti¬ 
vation  to  change. 

In  a  second  one,  the  same  randomized  ap¬ 
proach  was  used  with  a  sample  of  36  women 
with  BED.^^  The  resuits  showed  that  77%  of  the 
ECT  group  quit  binging  after  6  months  versus 
56%  for  the  CBT  sample  and  22%  for  the  nutri¬ 
tional  group  sample.  Moreover,  the  ECT  sample 
reported  better  scores  in  most  psychometric 
tests  including  EDI-2  and  body  image  scores. 

In  the  final  one,  recently  presented  in  the  Medi¬ 
cine  Meets  Virtual  Reality  Conference  2005,  lET 
was  compared  with  nutritional  and  cognitive- 
behavioral  treatments,  using  a  randomized  con- 
trolied  trial,  in  a  sample  of  211  female  obese 
patients.  Both  lET  and  CBT  produced  a  better 
weight  loss  than  NT  after  a  6-month  follow-up. 
However,  lET  was  able  to  significantly  improve, 
over  CBT  and  NT,  both  body  image  satisfaction 
and  self-efficacy.  This  change  produced  a  re¬ 
duction  in  the  number  of  avoidance  behaviors 
as  well  as  an  improvement  in  adaptive  behav¬ 
iors. 


Figure  1:  A  patient  is  trying  the  Integrated  Experiential  Therapy 
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The  Spanish  research  group  led  by  Cristina  Bo- 
tella  has  compared  the  effectiveness  of  VR  to 
traditional  CBT  for  body  image  (based  on 
Cash^^  and  reported  on  a  small  controlled  study 
with  a  clinical  population. 


The  main  tool  they  developed  is  a  3D  figure, 
experienced  through  an  immersive  headset, 
whose  body  parts  (arms,  thighs,  legs,  breasts, 
stomach,  buttocks,  etc.)  that  could  be  enlarged 
or  diminished.®  Further,  the  body  could  be 


VR  Workstation 


SGI  Onyx4  visualization  system  with  InfinitePerformance  &  lnfiniteReality4 
raphics 


SGI  Onyx  350,  V12  Graphic  Card,  2x400MHz  processors,  512  Mbyte  Ram, 
18  Gbyte  Hard  Disk 


Xeon  branded  PC,  2x3Ghz  processors,  1Gbyte  Ram,  2x200  Gbyte  Hard 
Disk  and  17”  monitor 


Pentium  IV  or  Athlon  XP  branded  PC,  3.4  Ghz  processor,  512  Mbyte  Ram, 
200  Gbyte  Hard  Disk  and  17”  monitor 


Consumer  graphic  cards 


Nvidia  GeForceFX  6800  Ultra  256  Mbyte  Vram  PCI  Express 


ATI  Radeon  X800  Pro  All-in-Wonder  256  Mbyte  PCI  Express 


Professional  graphic  cards 


Quadro4  FX  4400  512  Mbyte  Vram  PCI  Express 


ATI  FireGL  X3-256  256  Mbyte  Vram  AGP 


Tracking  system 


Polhemus  Fastrak 


Ascension  PC  Flock  of  Birds 


Intersense  Intertrax  2 


3D  Shutter  Glasses 


StereoEyes  Wireless 


Elsa  3D  Revelator  IR 


Head  Mounted  Display 


Kaiser  Proview  XL  40/50  (XGA  resolution  -  3D,  wide  fov) 


Daeyang  l-Visor  DH4400  VP  3D  (SVGA  resolution  -  3D) 


Olympus  Eye-Trek  FMD-700  (SVGA  resolution  -  2D) 


Daeyang  l-Visor  DH4400  VP  (SVGA  resolution  -  2D) 


Sony  Glasstron  PLM-A35  (Video  output  only  -  2D) 


VR  Gloves 


Pinch  Glove 


5DT  Right  Hand 


Indicative  Prices 
(as  01  Jan  05) 


250000  US$ 


18000  US$ 


4200  US$ 


2600  US$ 


550  US$ 


550  US$ 


1600  US$ 


1200  US$ 


7000  US$ 


2200  US$ 


1100  US$ 


320  US$ 


180  US$ 


50000  US$ 


1900  US$ 


1300  US$ 


1200  US$ 


500  US$ 


2000  US$ 


650  US$ 
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evaluated  wholly  or  in  parts  and  placed  in  differ¬ 
ent  contexts  (for  instance,  in  the  kitchen,  before 
eating,  after  eating,  facing  attractive  persons, 
etc.). 

Within  the  published  study,  five  patients  -  3  with 
Anorexia  Nervosa  and  2  with  Bulimia  Nervosa 
completed  the  traditional  treatment  and  8-4 
with  Anorexia  Nervosa  and  4  with  Bulimia  Ner¬ 
vosa  -  completed  the  VR  condition. 

Although  both  groups  showed  improvements  on 
general  ED  measures,  there  were  no  significant 
between-group  differences.  However,  the  VR 
group  did  show  greater  improvement  on  meas¬ 
ures  of  body  image,  dysphoria,  and  anxiety, 
leading  the  authors  to  conclude  that  VR  targets 
disturbances  in  body  image  better  than  stan¬ 
dard  CBT.  On  a  10-point  scale,  patients  rated 
the  realism  of  VR  from  7  to  9.  Since  then,  the 
group  has  also  developed  a  VR  simulator  of 
food  and  eating^®  actually  under  evaluation  with 
patients. 

CONCLUSIONS 

In  summary,  the  published  data  suggest  that  VR 
can  help  in  addressing  two  key  features  of  eat¬ 
ing  disorders  and  obesity  not  always  adequately 
addressed  by  existing  approaches:  body  experi¬ 
ence  disturbances  and  self-efficacy.  VR  tech¬ 
nology  offers  an  innovative  approach  to  the 
treatment  of  body  image  disturbance,  a  difficult 
concept  to  address  in  therapy.  Previously,  cog¬ 
nitive-behavioral  and  feminist  approaches  have 
been  the  standard  interventions,  although  in  our 
experience,  it  seems  that  many  patients  con¬ 
tinue  to  struggle  with  negative  body  image  post¬ 
treatment. 

As  emphasized  by  social  cognitive  theory,  per¬ 
formance-based  methods  are  the  most  effective 
in  producing  therapeutic  change  across  behav¬ 
ioral,  cognitive,  and  affective  modalities.^®  The 
proposed  experiential  approach  could  help  pa¬ 
tients  in  discovering  that  difficulties  can  be  de¬ 
feated,  so  improving  their  cognitive  and  behav¬ 
ioral  skills  for  coping  with  stressful  situations. 

On  the  downside,  a  limited  number  of  patients 
(less  than  2%)  experienced  simulator  sickness, 
which  consists  of  nausea,  disorientation,  and 
eye  strain  or  blurred  vision  during  and  after  use. 
In  addition,  the  cost  of  this  technology  remains 


relatively  high  (see  Table  1)  and  availability  is 
still  limited:  a  typical  VR  system  as  the  one  used 
in  the  studies  discussed  before  costs  between 
15000  and  25000  euro  (hardware  10000-15000 
euro;  software:  5000-10000  euro). 

Finally,  communication  networks  have  the  po¬ 
tential  to  transform  VEs  into  shared  worlds  in 
which  individuals,  objects,  and  processes  inter¬ 
act  without  regard  to  their  location.  In  the  next 
five  years,  such  networks  will  probably  merge 
VR  and  telemedicine  applications  allowing  us  to 
use  VR  for  such  purposes  as  distance  learning, 
distributed  training,  and  e-therapy. 
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Virtual  Technologies  for  Extreme  Environment  Effect  Mitigation 

E.  Gaia,  Dr.  Ing 
Alenia  Spazio  S.p.A.,  Torino  Italy 


Abstract:  This  paper  examines  the  demands  of  a  manned  system  operating  in  an  extreme  environ¬ 
ment,  where  there  is  the  need  to  enhance  or  at  least  maintain  crew  health,  team  spirit  and  cohesion, 
and  how  nouvelles  techniques  based  on  Virtual  Reality  technologies  can  provide  some  answers.  A 
typical  example  is  a  manned  mission  to  Mars  with  a  crew  of  4-8  astronauts  where  it  is  mandatory  to 
maintain  their  psychological  health  for  a  mission  that  is  envisaged  to  last  from  two  to  three  years. 
They  will  have  to  work  and  live  in  an  artificial  environment  with  limited  resources  (for  example  com¬ 
munication  capability  with  ground,  due  to  the  distances  involved,  cannot  be  a  direct  two  way  system) 
and  living  volumes.  The  paper  proposes  some  possible  fields  of  investigation.  It  will  also  consider  the 
fact  of  how  these  innovative  technologies  based  on  Virtual  Reality  will  have  to  be  developed  and 
tested. 


INTRODUCTION 

In  the  next  25  years  we  can  expect  to  see  two 
things.  First  human  beings  will  be  installed  in 
various  extreme  environments  of  which  one  can 
be  the  Moon  or  even  Mars.  On  the  other  hand 
many  new  Virtual  Reality  (VR)  applications  will 
be  available  to  augment  our  capabilities  in  vari¬ 
ous  areas  from  entertainment  to  medicine  and 
engineering,  but  it  is  also  possible  to  envisage 
that  VR  or  AR  (Augmented  Reality)  may  be  es¬ 
pecially  employed  where  the  human  being  is 
facing  highly  demanding  challenges.  In  this  pa¬ 
per  what  might  be  used  in  future  manned  space 
missions  is  discussed  with  a  special  emphasis 
on  the  possible  utilisation  of  VR/AR  to  support 
the  maintenance  of  on-board  crew  well-being. 

IDENTIFICATION  OF  THE  PROBLEM 

The  future  Moon  and  Mars  exploration  missions 
foresee,  after  some  pilot  missions  to  the  Moon, 
a  series  of  long  missions  with  human  perma¬ 
nence  on  board.  These  range  from  perma¬ 
nence  on  the  Moon  surface  of  one  year  (the  trip 
of  3-4  days  to  the  Moon  is  negligible  compared 
to  the  permanence)  to  a  mission  to  Mars  of 
three  years  of  which  up  to  one  may  be  spent  in 
travelling  to  and  fro.  The  envisaged  size  of  the 
crew  ranges  from  4  to  8  people,  each  of  whom 
is  highly  motivated  and  knowledgeable.  The 
crew  will  certainly  be  specifically  selected  and 
highly  trained.  In  addition,  they  will  receive  all 


possible  support,  and  it  is  expected  that  VR/AR 
techniques  will  be  widely  employed  to  provide 
this  support,  augmenting  crew  capability  in  con¬ 
trolling  the  overall  system,  robots  and  handling 
communications.  It  would  not  be  surprising,  for 
example,  if  VR/AR  based  training  techniques 
were  employed  both  on  ground  in  preparing  the 
mission  and  in  orbit  to  handle  the  unexpected. 
But  there  is  another  area  where  VR/AR  may  be 
employed,  that  is  in  the  maintenance  of  the 
crew’s  psychical  well-being  and  in  maintaining 
the  team’s  positive  attitude  and  cohesion. 


TYPICAL  MISSION:  EXPLORATION  SPIRAL 
3/LUNAR  BASE  AND  MARS  TESTBED 

From  NASA  Requirements  Document  “ESMD- 
RQ-0013”.  “Exploration  Spiral  3  will  establish 
the  capability  to  conduct  routine  human  long- 
duration  missions  at  a  lunar  base  to  test  out 
technologies  and  operational  techniques  for 
expanding  the  human  presence  to  Mars  and 
beyond.  Missions  in  Spiral  3  will  extend  up  to 
several  months  in  duration  at  the  lunar  poles  or 
equatorial  region  in  order  to  serve  as  an  opera¬ 
tional  analog  of  future  Mars  missions.  Spiral  3 
will  require  the  development  and  deployment  of 
habitats  and  surface  power  systems.  ...  Once 
the  surface  systems  are  in  place,  successively 
longer  missions  will  be  conducted  to  increase 
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the  understanding  of  system  technical  perform¬ 
ance  (including  health  and  human  systems), 
and  to  provide  increasing  levels  of  operational 
autonomy  capabilities  that  will  be  necessary  for 
future  human  Mars  exploration  missions.” 

Related  Main  Mission  Characteristics: 

•  Crew:  4  Astronauts. 

•  Gravity:  1/6  of  the  gravity  of  the  Earth 

•  Mission  Duration: 

o  6  days  travel  from  Earth  to  Moon; 

o  42  to  600  days  permanence  on  the  sur¬ 
face; 

o  7.5  days  from  Moon  back  to  Earth. 

•  Habitation  Facilities: 

o  Transportation  phases  Crew  Exploration 
Vehicle  (CEV)  plus  Mission  Module 
approx.  14  m^  volume  available  for  the 
crew; 

o  Up  to  200  m^  gross  volume  available  for 
the  crew  for  habitability  and  working  areas. 
This  volume  may  be  increased  with  an 
additional  logistic  cargo  module  that  can 
be  connected  to  the  main  habitation  and 
laboratory  module. 

•  Surface  Crew  working  and  related  Activi¬ 
ties: 

o  It  is  expected  that  in  one  year  there  will  be 
more  than  200  Extra  Vehicular  Sorties  with 
two  people  in  each  one; 

o  Control  the  extensive  utilization  of  robots 
and  automated  Rovers  for  exploration 

o  Other  activities  foreseen  are  system  main¬ 
tenance  and  performance  of  experiments 
to  prepare  for  Mars  missions. 

•  Surface  Crew  off-duty  and  related  Activities: 

o  Housekeeping 

o  Exercise  &  health  maintenance 


o  Eating,  sleeping  and  hygiene 
Free  time. 

Identified  issues: 

•  Communications:  although  the  distance 
between  the  Moon  and  the  Earth  is  relatively 
small,  communication  delay  is  of  approximately 
6  seconds  each  leg.  This  will  require  a  different 
concept  of  control  for  all  used  automated  sys¬ 
tems.  Now  most  of  the  controls  are  performed 
on  ground  by  the  flight  control  personnel.  For 
voice  communication  it  might  be  only  a  nui¬ 
sance  but  its  effects  on  the  crew  over  a  long 
period  are  unknown.  In  case  of  Mars  mission 
the  communication  delay  can  be  up  to  20  min¬ 
utes  each  leg:  this  will  sharply  increase  the 
communication  problem. 

•  Safety  &  Isolation:  the  handling  of  emer¬ 
gencies  during  these  exploration  missions  is 
completely  innovative  with  respect  to  the  ap¬ 
proach  taken  for  the  Space  Station  where  an 
injured  or  ill  crew  member  can  be  back  on  the 
Earth  in  approximately  12-16  hours.  Starting 
from  the  Moon  surface  it  would  take  as  a  mini¬ 
mum  5-6  days  and  from  Mars  up  to  6  months  to 
return  to  Earth,  so  most  of  the  emergencies 
must  be  handled  there. 

•  Group  dynamics  and  roles  of  each  astro¬ 
naut:  this  is  one  of  the  keys  to  mission  suc¬ 
cess,  which  means  being  able  to  maintain  - 
throughout  the  mission  and  despite  any  prob¬ 
lem  that  may  arise  -  a  positive  attitude  of  each 
single  component  of  the  team  as  well  as  of  the 
team  itself. 

POSSIBLE  ROLE  OF  VR 

Most  probably  there  will  be  quite  ample  applica¬ 
tions  of  VR/AR  techniques  to  improve  the  qual¬ 
ity  of  the  working  environment  on  board  during 
these  long  exploration  missions.  These  applica¬ 
tions  can  be  envisaged  in  the  field  of: 

•  Communications  between  Moon  base  and 
the  Earth  or  between  the  exploring  astronauts 
and  the  Moon  Base. 

•  Training  (on  the  job) 

•  Automatics  and  Robotics  command/control 
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•  Support  to  procedure  implementation, 
Leisure  i.e.  movies  and  scenery  display  (some 
students  have  conceived  a  relaxation  room  with 
VR) 

All  these  applications  are  up  to  now  perceived 
as  single  pieces  of  application,  each  developed 
to  handle  a  specific  need  but  not  linked  one  to 
the  other.  What  has  been  missing  up  to  now  is 
an  integrated  system  that  can  perform  all  that 
has  been  mentioned  before,  but  beyond  that 
will  be  able  to  take  care  of  the  maintenance  of 
the  psychological  well-being  of  the  astronauts 
both  at  individual  level  as  well  as  at  team  level. 
This  system  shall  be  capable  to  move  from  pre¬ 
sent  applications  of  VR  techniques  aimed  at 
restoring  people’s  well-being  by,  for  example, 
treating  the  fear  of  flying  like  the  VR  technique 
described  in  reference^  to  a  nouvelle  application 
of  monitoring  and  maintaining  the  human  be¬ 
ing’s  well  being. 

An  integrated  system  based  on  VR/AR  tech¬ 
nique  of  exchange  of  information  and  control 
with  the  crew  also  with  the  use  of  nouvelle  tech¬ 
niques  of  Brain  Computer  Interface  (BCI)  can 
be  developed.  This  integrated  system  can  not 
only  give  the  crew  an  effective  support  both  in 
working  and  non-working  activities  but,  since  it 
can  be  envisaged  that  it  would  be  capable  of 
monitoring  their  health,  it  can  also  intervene  by 
taking  actions  to  reduce/control  potentially  dan¬ 
gerous  situations. 

DISCUSSION 

Clearly  a  system  such  as  the  one  proposed  is 
not  available  in  the  short  term  and  it  will  take  a 
well  coordinated  effort  to  develop;  on  the  other 
hand,  there  are  some  issues  -  some  simple  but 
some  others  much  more  complex  involving  eth¬ 
ics  -  that  it  is  better  to  start  discussing  now. 

The  following  are  the  issues  to  be  investigated 
in  order  to  set  the  requirements  capable  to  de¬ 
scribe  a  system  to  be  produced  that  respects 
the  ethics  of  the  human  being  but  on  the  other 
hand  is  capable  of  guaranteeing  the  mission 
success: 

•  Private  experiences  vs.  common  experi¬ 
ence  (screen  vs.  helmet  or  both):  How  can  the 
group  dynamics  be  supported? 


•  Communication  system:  How  to  overcome 
the  time  lag  constraint  maintaining  a  level  of 
“quality”  and  “effectiveness”  in  communication? 
-  especially  if  it  is  used  as  a  keystone  for  well¬ 
being  maintenance. 

•  Psychological  Health  maintenance:  How 
and  when  can  VR/AR  be  used  to  support  the 
Crew’s  Psychological  Health? 

•  Psychological  Health  monitoring  system: 
How  can  the  psychological  well-being  of  the 
singular  member  and  of  the  group  be  moni¬ 
tored? 

•  Training  and  other  VR/AR  applications  used 
for  Crew  support:  VR/AR  are  recognised  candi¬ 
date  techniques  for  training  as  well  as  to  pro¬ 
vide  information  and  control  capability  to  a  crew 
member  but  can  this  VR/AR  system  be  ex¬ 
ploited  to  the  level  required  without  overloading 
the  human  being? 

•  Human  VR  interfaces:  New  interfaces  are 
being  developed  including  the  BCI:  how  can 
these  be  used  without  violating  ethical  rules? 

If  such  a  system  can  be  developed  and  ac¬ 
cepted  for  utilisation  would  it  be  possible  to  re¬ 
fer  to  it  as  VR/AR  system  or  must  we  use  a  new 
word  like  an  enhanced  reality  system? 

CONCLUSION 

This  paper  was  not  aimed  at  giving  solutions 
but  more  at  collecting  questions  and  doubts. 
Nevertheless  two  positive  issues  can  be  gath¬ 
ered  from  what  was  presented  before: 

•  First  how  extreme  environments  can  be 
used  as  technological  trend  identification  and 
then  can  become  valid  test  beds  for  research 
and  development  of  new  techniques 

How  a  user  centred  approach  is  more  and  more 
needed  in  order  not  to  be  overcome  by  the 
technological  push  especially  in  new  areas  like 
VR/AR. 
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Abstract;  VR  applications  in  psychotherapy  are  gaining  prominence  in  the  research  community  and 
acceptance  among  practitioners,  with  particular  successes  in  the  area  of  phobia  treatments.  How¬ 
ever,  a  promising  technology  remains  unexplored:  computational  models  of  cognition  and  emotion. 
Cognitive  models  (also  termed  cognitive-  or  agent-architectures)  aim  to  emulate  cognitive  processing 
such  as  attention,  perception,  and  decision-making,  and  are  used  by  cognitive  scientists  to  advance 
understanding  of  the  mechanisms  and  structures  mediating  cognition.  These  models  are  also  used  in 
applied  settings  to  improve  the  realism  of  training  and  assessment  environments,  and  to  improve 
human-computer  interaction  and  system  design.  Recently,  architectures  have  been  developed  that 
explicitly  represent  emotions:  both  emotion  appraisal  processes,  and  effects  of  emotions  on  cogni¬ 
tion.  Such  computational  models  of  cognition  and  emotion  have  two  types  of  applications  in  cy¬ 
bertherapy.  First,  they  can  enhance  the  realism  of  a  synthetic  agents  used  in  VR  assessment  and 
treatment  environments  (e.g.,  avatars  used  in  social  phobia  treatment).  Second,  they  have  the  poten¬ 
tial  to  advance  our  understanding  of  the  etiology  and  treatment  of  a  variety  of  affective  disorders,  by 
enabling  the  modeling  of  the  mechanisms  of  cognitive-affective  interactions  that  play  central  role  in 
these  disorders,  and  in  their  treatments.  In  this  paper  I  first  describe  a  cognitive-affective  architec¬ 
ture  capable  of  modeling  the  dynamic  generation  of  emotions  (affect  appraisal),  and  selected  effects 
of  emotion  on  cognition.  I  then  describe  possible  applications  of  this  architecture  to  psychotherapy 
practice  and  research. 


INTRODUCTION 

VR  applications  in  psychotherapy  are  gaining 
prominence  in  the  research  community  and  ac¬ 
ceptance  among  practitioners.  Significant  suc¬ 
cess  has  been  achieved  with  VR  therapies 
aimed  at  a  variety  of  phobias  and  PTSD  (e.g., 
httD://www.  virtuallvbetter.com/).''  However,  a 
promising  technology  remains  unexplored;  com¬ 
putational  models  of  cognition  and  emotion. 
Cognitive  models  (also  termed  cognitive-  or 
agent-architectures)  aim  to  emulate  cognitive 
processing  such  as  attention,  perception,  learn¬ 
ing,  problem-solving,  planning  and  decision¬ 
making,  and  are  used  by  cognitive  scientists  to 
advance  understanding  of  the  mechanisms  and 
structures  mediating  cognition.^  These  models 
are  also  used  in  applied  settings  to  improve 
training  and  human-system  design.^  Recently, 
architectures  have  been  developed  that  explic¬ 
itly  represent  emotions;  both  emotion  appraisal 
processes,  and  effects  of  emotions  on  cogni¬ 


tion. 
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While  a  number  of  researchers  are  advocating 
the  use  of  avatars  in  VR  therapy  environ- 
ments,^°  and  promising  research  efforts  exist,^^ 
little  work  has  been  done  in  coupling  synthetic 
avatars  with  cognitive-affective  architectures. 
Thus,  the  potential  of  cognitive  architectures, 
and  associated  computational  modeling  re¬ 
search  methods,  have  not  been  explored  in  psy¬ 
chotherapy  research  and  practice.  The  primary 
purpose  of  this  paper  is  to  introduce  computa¬ 
tional  cognitive-affective  architectures  to  the 
cybertherapy  community,  and  describe  exam¬ 
ples  of  possible  applications  in  psychotherapy 
research  and  practice.  These  include  the  en¬ 
hancement  of  synthetic  avatars  in  VR  treatment 
and  assessment  environments  (e.g.,  distinct 
avatars  could  be  defined  to  support  dynamic, 
adaptive  interaction  during  social  phobia  treat¬ 
ment,  each  with  different  behavioral  characteris¬ 
tics  to  practice  particular  situations),  and  the 
development  of  computational  models  aimed  at 
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advancing  our  understanding  of  the  mecha¬ 
nisms  that  contribute  to  the  development  of 
emotional  disorders  (e.g.,  inappropriate  coping 
strategies),  and  those  involved  in  treatment 
(e.g.,  cognitive  restructuring).  The  paper  is  or¬ 
ganized  as  follows.  First,  I  describe  a  cognitive- 
affective  architecture  capable  of  modeling  the 
dynamic  generation  of  emotions  (affect  ap¬ 
praisal),  and  some  effects  of  emotions  on  cog¬ 
nition  (section  2).  I  then  discuss  how  this  archi¬ 
tecture  could  enhance  psychotherapy  practice 
(section  3.1)  and  how  it  could  be  used  to  ad¬ 
vance  research  in  the  area  of  affective  disor¬ 
ders,  by  enabling  the  modeling  of  the  mecha¬ 
nisms  of  the  cognitive-affective  interactions  me¬ 
diating  both  the  development  and  treatment  of  a 
range  of  affective  disorders  section  3.2). 

MAMID  COGNITIVE-AFFECTIVE  ARCHITEC¬ 
TURE 

The  MAMID  cognitive-affective  architecture  is 
an  integrated  symbolic  architecture,  aimed  at 
emulating  aspects  of  human  information  proc¬ 
essing,  with  particular  focus  on  the  role  of  emo¬ 
tion  in  decision-making  (see  figure  1A).  To  this 
end,  MAMID  models  the  cognitive  appraisal 
process  to  dynamically  generate  emotions  in 
response  to  incoming  stimuli,  and  then  models 
the  subsequent  effects  of  these  emotions  on 
distinct  stages  of  decision-making.  Below  we 
describe  the  MAMID  architecture,  the  generic 
methodology  for  modeling  effects  of  emotions 
(and  other  individual  differences,  including  per¬ 
sonality  traits),  and  a  preliminary  evaluation. 


MAMID  ARCHITECTURE 

MAMID  implements  a  sequential  see-think-do 
processing  sequence  (figure  IB),  consisting  of 
the  following  modules:  sensory  pre-processing, 
translating  incoming  data  into  task-relevant 
cues;  attention,  filtering  incoming  cues  and  se¬ 
lecting  a  subset  for  processing;  situation  as¬ 
sessment,  integrating  individual  cues  into  an 
overall  situation  assessment;  expectation  gen¬ 
eration,  projecting  current  situation  onto  possi¬ 
ble  future  states;  affect  appraiser,  deriving  the 
affective  state  (both  valence  and  four  of  the 
basic  emotions)  from  a  variety  of  external  and 
internal  elicitors,  both  static  and  dynamic;  goal 
selection,  selecting  critical  goals  for  achieve¬ 
ment;  and  action  selection,  selecting  the  best 
actions  for  goal  achievement.  These  modules 
translate  the  incoming  stimuli  (cues)  onto  the 
visible  behavior  (actions),  via  a  series  of  inter¬ 
mediate  internal  representational  structures 
(situations,  expectations,  and  goals),  collec¬ 
tively  termed  mental  constructs. 

This  “translation”  is  enabled  by  long-term 
memories  (LTM)  associated  with  each  module, 
represented  in  terms  of  belief  nets  or  rules. 
Belief  nets  resemble  causal  graphs  and  are 
well-suited  for  representing  causal  relationships 
among  propositions,  and  at  manipulating  uncer¬ 
tainty.  Rules  represent  ‘IF-THEN’  relationships, 
and  are  well-suited  for  representing  simple  in¬ 
ferences.  (Figure  4  shows  examples  of  rules.) 
Mental  constructs  are  characterized  in  terms  of 
a  number  of  attributes  (e.g.,  familiarity,  novelty. 
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Figure  1.  (A)  Modeling  Affect  Appraisal  and  Emotion  Effects  within  a  Cognitive  Architecture; 
(B)  MAMID  Cognitive  Architecture:  Modules  &  Mental  Constructs 
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salience,  threat  level,  valence,  etc.),  which  col¬ 
lectively  determine  their  processing:  that  is,  the 
likelihood  of  a  cue  will  be  attended,  situation 
derived,  goal  or  action  selected,  etc. 

The  Affect  Appraisal  module  plays  a  central  role 
in  MAMID  (see  figure  2).  It  incorporates  ele¬ 
ments  of  several  recent  appraisal  theories:  mul- 
f/p/e-levels'^'’^'^®  and  multiple  stages.’'^  The 
multi-level  structure  generates  both  a  low- 
resolution  assessment  of  the  current  set  of 
stimuli,  in  terms  of  a  valence,  and  a  higher- 
resolution  categorical  assessment,  in  terms  of 
four  of  the  basic  emotions:  anxiety/fear,  anger, 
sadness,  happiness.  Its  multi-stage  structure 
uses  both  universal  elicitors  (e.g.,  novelty, 
threat  level,  pleasantness,  unexpectedness),  to 
generate  the  valence  using  an  automatic  ap¬ 
praisal  (roughly  corresponding  to  the  largely 
‘hardwired’,  ‘primitive’  appraisal  components), 
and  more  cognitively-complex  and  idiosyncratic 
elicitors  (e.g.,  individual  history, 
expectation-  and  goal-congruence),  to  generate 
a  categorical  assessment  using  an  expanded 
appraisal. 

The  resulting  affective  states  then  influence 
processing  in  several  ways:  (1)  they  are  used 
directly  in  the  rules  selecting  the  agent’s  goals 
and  actions:  (2)  they  influence  the  speed  and 
capacity  of  the  architecture  modules:  (3)  they 
influence  mental  construct  ranking,  thus  deter¬ 
mining  whether  a  specific  cue  or  situation  is 
processed,  or  specific  goal  selected.  The  last 
two  effects  have  been  a  particular  focus  of  this 
effort,  and  aim  to  emulate  some  of  the  empiri¬ 
cally-identified  mechanisms  of  emotion  effects 


within  the  perceptual  and  cognitive  apparatus, 
as  outlined  above. 

GENERIC  STATE  AND  TRAIT  MODELING 
METHODOLOGY 

MAMID  uses  a  previously  described  methodol¬ 
ogy  for  modeling  state  and  trait  effects  within  a 
cognitive  architecture,®'^®  which  consists  of 
mapping  particular  state  /  trait  profiles  (e.g., 
high  trait  and  state  anxious  individual)  onto  spe¬ 
cific  architecture  parameter  values  (figure  3). 
These  parameters  then  control  processing 
within  individual  architecture  modules.  Func¬ 
tions  implementing  these  mappings  were  con¬ 
structed  on  the  basis  of  the  available  empirical 
data.  For  example,  reduced  attentional  and 
working  memory  (WM)  capacity,  associated 
with  anxiety  and  fear,  are  modeled  by  dynami¬ 
cally  reducing  the  attentional  and  WM  capacity 
of  the  architecture  modules,  which  then  reduces 
the  number  of  constructs  processed  (fewer 
stimuli  attended,  situations  derived,  expecta¬ 
tions  generated,  etc.).  Attentional  threat  bias  is 
modeled  by  higher  ranking  of  threatening  cues, 
thus  increasing  their  likelihood  of  being  at¬ 
tended,  and  by  higher  ranking  of  threatening 
situations  and  expectations,  thus  increasing  the 
chances  of  a  threatening  situation  /  expectation 
being  derived.  Trait-linked  structural  differ¬ 
ences  in  LTM  are  supported  by  allowing  the 
flexible  selection  of  alternative  LTM  clusters, 
reflecting  distinct  personality  traits  (e.g.,  selec¬ 
tion  of  clusters  with  greater  proportion  of  threat- 
and  self-related  schemas  to  represent  individu¬ 
als  with  high  trait-anxiety  (high  neuroticism). 
Traits  also  influence  the  dynamic  characteristics 
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Figure  3.  (A)  Parametric  State /Trait  Modeling  Methodology; 

(B)  Modeling  Threat-Bias  Effects  Associated  with  Trait  and  State  Anxiety 


of  the  emotional  responses  (ramp  up,  decay, 
and  maximum  intensities). 

There  are  several  advantages  of  this  methodol¬ 
ogy  for  state-trait  effect  modeling:  (1)  it  facili¬ 
tates  rapid  modeling  of  a  broad  range  of  distinct 
individual  profiles;  (2)  the  rich  architecture 
parameterization  allows  the  definition  of  addi¬ 
tional  high-level  individual  characteristics  (e.g., 
obsessive-compulsive);  (3)  it  provides  a  means 
of  integrating  the  possibly  conflicting  effects  of 
multiple,  interacting  traits  and  states,  much  as 
these  influences  interact  in  humans. 

RESULTS 

Feasibility  of  the  model  was  demonstrated  in 
the  context  of  a  simulated  peacekeeping  sce¬ 
nario,  where  separate  instances  of  the  architec¬ 
ture  controlled  the  behavior  of  ‘stereotypical’ 
unit  leaders  (‘anxious’,  ‘aggressive’,  ‘normal’). 
The  same  set  of  external  stimuli  triggered  dis¬ 
tinct  emotions  and  emotional  response  patterns 
in  the  different  stereotypes  (e.g.,  higher  anxiety 
levels  in  the  ‘high  anxious’  stereotype),  and 
their  effects  on  decision-making  then  caused 
differences  in  observable  behaviors  (e.g., 
slower  movement  and  more  communication 
behaviors  in  the  anxious  stereotype).  The  MA- 
MID  architecture  is  domain-independent,  to  fa¬ 


cilitate  transitions  to  other  domains,  including 
psychotherapy. 

APPLICATIONS  OF  COGNITIVE-AFFECTIVE 
ARCHITECTURES  IN  PSYCHOTHERAPY 

Below  we  describe  how  the  MAMID  cognitive- 
affective  architecture  could  be  used  to  enhance 
the  effectiveness  of  therapeutic  VR  environ¬ 
ments,  as  well  as  how  it  could  advance  our  un¬ 
derstanding  of  the  etiology  and  treatment  of  a 
range  of  affective  disorders,  by  supporting 
adaptive,  dynamic  interaction  between  patients 
and  synthetic  avatars. 

Applications  in  Clinical  Practice 

MAMID’s  ability  to  represent  a  variety  of  distinct 
stereotypes  (in  terms  of  distinct  state-trait  pro¬ 
files),  and  to  control  the  behavior  of  synthetic 
avatars  in  virtual  environments,  enables  it  to 
generate  a  broad  range  of  avatar  stereotypes, 
characterized  by  distinct  behavioral  repertoires 
(e.g.,  ‘aggressive  audience  member’,  ‘angry 
friend’,  ‘anxious  spouse’).  The  ability  to  dynami¬ 
cally  generate  individual  idiosyncratic  behavior 
characteristic  of  these  stereotypes  then  en¬ 
hances  the  avatar  believability,  and  effective¬ 
ness  within  therapeutic  VR  environments. 

For  example,  to  treat  social  phobia,  or  fear  of 
public  speaking,  a  virtual  environment  would  be 
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.  Rule'  Respond  to  Spiater  ;  ; 

IF  (Situation  ~  Speaker  talking) 
^upportivdDAudience  p- 
Member  Avatar  ...JHEN 

"  (Action  -  Facial  expression:  smile) 
(Action  =  Head  movement:  nod) 
(Action  =  Utterance:  IJhat®  rightC)) 


Aggressive  Audience 
Member  Avatar 


Rule ‘Respoiid  to  Speaker 

IF  (Situation  =  Speaker  talking) 

THEN  " 

(Action  =  Facial  expression:  frown) 
(Action  =  Head  movement:  shake) 
(Action  =  Utterance:  IJ  donOthink  soO) 


Figure  4.  Examples  of  MAMID  Rules  Controlling  Action  Selection  in  Two  Distinct  Stereo¬ 
types,  Controlling  Avatars  in  a  “Fear  of  Public  Speaking”  Application 


populated  by  synthetic  avatars  displaying  par¬ 
ticular  characteristics  that  trigger  undesirable 
symptoms  (e.g.,  aggressive  individuals  would 
be  defined  for  patients  exhibiting  fear  of  public 
speaking  symptoms  when  confronted  with  criti¬ 
cal  or  aggressive  remarks).  The  avatars  would 
then  generate  aggressive  or  critical  behaviors 
(e.g.,  frown  while  producing  utterances  critical 
of  the  speaker-patient).  Figure  4  shows  exam¬ 
ples  of  two  rules  that  would  define  distinct  be¬ 
havior  sequences  for  distinct  audience  member 
avatars.  The  ‘optimal’  behaviors  triggering  the 
undesirable  symptoms  would  be  identified  em¬ 
pirically,  so  that  the  avatar  behavior  could  be 
customized  to  produce  the  desired  level  of  anxi¬ 
ety  in  the  patient.  Once  identified,  the  patient 
would  be  exposed  to  these  behaviors  to  imple¬ 
ment  the  desired  therapeutic  protocol  (e.g.,  ex¬ 
posure  therapy,  systematic  desensitization). 
The  patient  would  be  able  to  interact  with  the 
avatars  (e.g.,  defend  himself  against  a  verbal 
attack,  take  time  to  implement  a  coping  strat¬ 
egy),  and  the  avatar’s  ability  to  dynamically  re¬ 
spond  to  these  interactions  (e.g.,  back  off  vs. 
persist)  would  then  provide  a  degree  of  realism 
and  customization  that  are  not  currently  possi¬ 
ble  in  VR  treatment  environments.  The  use  of 
such  avatars,  embedded  within  simulated  situa¬ 
tions,  would  also  enhance  assessment. 

ADVANCING  PSYCHOTHERAPY  RESEARCH 
THROUGH  COGNITIVE-AFFECTIVE  MODEL¬ 
ING 

By  enabling  the  construction  of  computational 
causal  models  of  cognitive-affective  interac¬ 


tions,  the  MAMID  architecture  also  provides  a 
tool  for  modeling  the  etiology  and  treatment  of  a 
variety  of  disorders,  as  well  as  the  systematic 
development  of  targeted  assessment  and  treat¬ 
ment  protocols.  In  this  case  MAMID  would  be 
used  to  model  specific  cognitive-affective  proc¬ 
esses  characteristic  of  a  particular  disorder.  For 
example,  the  positive  feedback  cycle  among 
symptoms  and  behaviors  characterizing  gener¬ 
alized  anxiety  disorders  and  phobias  could  be 
modeled  by  representing  the  increasing  pre¬ 
dominance  of  anxiety-related  schemas  in  the 
patient’s  long-term  memory,  increased  sensitiv¬ 
ity  to  anxiety-producing  stimuli,  and  generaliza¬ 
tion  across  previously  neutral  stimuli,  as  the 
disorder  becomes  established,  all  contributing 
to  increasingly  avoidant  behaviors  and  with¬ 
drawal,  characteristic  of  generalized  anxiety. 
Results  of  therapeutic  interventions  could  then 
be  modeled  by  ‘exposing’  the  MAMID  architec¬ 
ture  patient  model  to  repeated  simulated  inputs, 
representing  particular  therapeutic  interventions 
(e.g.,  cognitive  restructuring,  systematic  desen¬ 
sitization),  as  well  as  changing  social  and  envi¬ 
ronmental  contexts.  This  would  then  result  in 
the  gradual  emergence  of  more  adaptive  mem¬ 
ory  schemas  and  interpretations  (in  the  case  of 
cognitive  restructuring  aimed  at  teaching  spe¬ 
cific  anxiety-coping  strategies),  and  diminished 
sensitivity  and  reactiveness  to  previously  anxi¬ 
ety-producing  stimuli  (in  the  case  of  systematic 
desensitization  and  exposure  protocols).  Im¬ 
proved  understanding  of  these  processes  would 
provide  opportunities  for  more  finely-tuned  as¬ 
sessment  and  treatment  protocols.  Compute- 
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tional  models  support  the  development  of  more 
refined  theories  and  the  generation  of  specific 
experimental  hypotheses  aimed  at  validating  a 
particular  theoretical  model. 

SUMMARY  AND  CONCLUSIONS 

While  much  progress  has  been  made  in  virtual 
reality  treatment  environments,  the  use  of  cog¬ 
nitive  models  to  control  synthetic  avatar  behav¬ 
ior  has  not  been  explored.  In  this  paper  I  sug¬ 
gest  that  the  incorporation  of  such  models 
would  greatly  enhance  the  effectiveness  of 
these  environments,  by  enhancing  the  believ- 
ability  and  effectiveness  of  synthetic  avatars, 
and  by  supporting  the  development  of  custom¬ 
ized  treatment  and  assessment  protocols. 

I  described  a  cognitive-affective  computational 
architecture  (MAMID),  capable  of  controlling  the 
behavior  of  such  adaptive  avatars.  MAMID 
models  affect  appraisal  processes  and  the  ef¬ 
fects  of  several  emotions  on  distinct  aspects  of 
decision-making,  thereby  enabling  the  definition 
of  a  variety  of  ‘stereotypes’,  capable  of  exhibit¬ 
ing  distinct  patterns  of  behavior  within  the  VR 
environments.  1  provided  examples  of  how  the 
MAMID  architecture  could  be  used  to  enhance 
believability  of  synthetic  avatars,  and  thereby 
enable  the  construction  of  customized  VR  treat¬ 
ment  and  assessment  environments.  I  also  dis¬ 
cussed  how  a  computational  model  of  cogni¬ 
tive-affective  interactions,  and  their  role  in  per¬ 
ception  and  decision-making  (including  situation 
assessment,  expectation  generation  and  goal 
management),  could  be  used  to  advance  our 
understanding  of  the  role  of  these  mechanisms 
in  the  etiology,  maintenance  and  treatment  of  a 
variety  of  affective  disorders.  To  this  end,  I  de¬ 
scribed  an  example  of  how  MAMID  could  be 
used  to  model  a  patient  suffering  from  a  particu¬ 
lar  set  of  symptoms,  and  how  specific  interven¬ 
tions  could  be  simulated  and  represented  within 
the  MAMID  model. 

The  development  of  cognitive  architecture  in 
general,  and  cognitive-affective  architectures  in 
particular,  is  an  active  research  area.  Much  pro¬ 
gress  has  been  made  in  modeling  specific  cog¬ 
nitive  processing  (e.g.,  acquisition  of  particular 
arithmetic  skill,  perception).  While  currently 
there  is  no  validated  architecture  of  cognition, 
the  existing  architectures  are  capable  of  pro¬ 


ducing  a  range  of  ‘stereotypical’  behaviors, 
which  would  resemble  real  humans  and  en¬ 
hance  VR  treatment  protocols  requiring  social 
interaction.  The  computational  modeling  meth¬ 
ods  are  also  well  suited  for  advancing  research 
in  clinical  psychology,  by  allowing  the  develop¬ 
ment  of  models  of  particular  disorders,  and  ex¬ 
ploring  the  different  effects  of  alternative  treat¬ 
ment  protocols. 

Research  programs  combining  empirical  stud¬ 
ies  with  computational  modeling  are  becoming 
more  common  in  cognitive  science.  I  believe 
that  the  modeling  methods  have  matured  to  the 
point  where  similar  applications  in  psychother¬ 
apy  research  can  be  of  great  benefit. 
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Immune  Attack:  Building  Biologically  Based  Immune  System  Simula¬ 
tions  for  Education  and  Training 

K.  Howell,  B.A.,  MBA 

Federation  of  American  Scientists,  Washington,  DC 


Abstract:  A  growing  literature  suggests  that  simulation,  visualization  and  gaming  can  play  an  impor¬ 
tant  role  in  deepening  understanding  of  difficult  concepts  in  mathematics,  engineering,  and  science. 
Learning  sciences  research  suggests  that  learning  by  doing  with  understanding  produces  better 
transfer  than  mere  doing  alone. Challenge-based  simulations  can  provide  students  opportunities  to 
receive  feedback  and  revise  their  thinking,  a  critical  part  of  the  learning  process.^  '^  Immune  Attack  is 
a  simulation  game  to  teach  biological  concepts  related  to  immunology  and  wound  infection,  allowing 
the  student  to  explore  the  internal  compartments  and  cells  of  the  human  body  and  visualize  immu¬ 
nological  processes.  Simulations  of  biological  functions  at  a  variety  of  levels,  from  subcellular  to  or¬ 
gan  systems,  that  are  easily  navigable  by  instructors  and  students  with  a  range  of  backgrounds  will 
provide  a  rich  exploratory  environment  for  learning  immunology. 


BACKGROUND 

Immune  Attack  is  a  simulation  game  to  teach 
biological  concepts  related  to  immunology  and 
wound  infection,  allowing  the  student  to  explore 
the  internal  compartments  and  cells  of  the  hu¬ 
man  body  and  visualize  immunological  proc¬ 
esses,  including  pathology.  The  game  com¬ 
bines  three-dimensional  visualizations  of  bio¬ 
logical  structure  and  function  with  advanced 
educational  technologies  to  provide  an  introduc¬ 
tion  to  basic  concepts  in  immunology  for  high 
school  students.  Immune  Attack  is  intended  to 
be  as  fun  and  compelling  as  the  computer 
games  currently  played  by  many  adolescents 
and  young  adults.  Students  are  motivated  with  a 
series  of  progressively  more  difficult  challenges 
in  a  gaming  environment  in  which  success  de¬ 
pends  on  increasingly  sophisticated  grasp  of 
concepts  in  immunology.  The  learning  experi¬ 
ences  are  individualized  by  use  of  context- 
sensitive  help  and  dialogues  and  continuous 
assessment  techniques  to  determine  when  the 
learner  is  ready  to  move  to  a  new  level.  The 
project  is  funded  by  a  National  Science  Founda¬ 
tion  Information  Technology  Research  grant. 

The  goal  of  this  research  is  to  test  the  hypothe¬ 
sis  that  visualization  of,  and  immersion  in  realis¬ 
tic  depictions  of  human  biology  will  be  highly 
engaging  to  high  school  students  and  hold  their 
interest  long  enough  for  learning  to  occur.  Ad¬ 


vances  in  cognitive  science  point  to  new  strate¬ 
gies  for  increasing  the  productivity  and  depth  of 
learning  if  the  learner  is  continuously  given  chal¬ 
lenges  whose  solution  depends  on  mastering 
the  material.^  The  game-based  simulations  that 
are  integral  to  the  gameplay  will  let  learners  ap¬ 
ply  their  knowledge  immediately  in  an  attempt  to 
defeat  a  variety  of  diseases,  from  simple  infec¬ 
tions  to  more  challenging  infections.  The  stu¬ 
dent’s  success  in  designing  an  immune  strategy 
capable  of  overcoming  an  array  of  infectious 
agents  will  provide  a  much  richer  measure  of 
understanding  than  standardized  tests.  Learn¬ 
ing  is  further  enhanced  if  the  students  are  so 
motivated  to  meet  the  challenges  that  they  are 
eager  to  ask  questions  that  can  help  them  -  and 
if  they  are  able  to  get  timely,  accurate  answers 
tailored  to  the  student  and  the  context  of  the 
question.® 

METHOD/TOOLS 

The  biological  models  for  the  game  were  devel¬ 
oped  working  closely  with  prominent  immunol¬ 
ogy  researchers  and  educators.  Experienced 
video  game  developers  are  developing  the 
game  and  assisting  in  integrating  the  learning 
tools.  The  learning  objectives  and  instructional 
strategies  are  being  developed  in  consultation 
with  biology  teachers,  at  both  the  high  school 


59 


Annual  Review  of  CyberTherapy  and  Telemedicine 


LEARNING  OBJECTIVES 

GAME  DESIGN/PLAY 

General 

The  student  will  comprehend  the  basic 
strategies  of  ma,jor  pathogens 

Simple  rules  govern  the  stylized  but  accurate  behaviors  of  a  variety 
of  bacteria,  viruses,  and  toxins. 

The  student  will  be  able  to  identify  and 
understand  role  of  key  components  of  the 
immune  system  including  innate  responses, 
inflammatory  responses,  and  secondary 
response  systems 

The  player  must  master  the  rules  that  govern  the  behavior  of  macro¬ 
phages,  neutrophils,  mast  cells,  NK  cells,  T  and  B  cells  and  will  be 
able  to  guide  these  cells  by  following  key  signals,  such  as  proteins. 

The  student  will  have  knowledge  of  how 
the  immune  systems  and  its  component 
results  clinical  symptoms 

Pop  up  tutorials,  a  health  status  bar,  and  visualizations  of  the  patient 
provide  clinical  information  to  alert  the  player  as  to  the  patient’s 
ohvsiolouic  condition  and  introduce  medical  terminolocv. 

The  student  will  be  knowledge  about  de¬ 
fects  in  the  immune  system 

Allergies,  autoimmune  disease,  AIDS  are  included  to  show  the  how 
the  disease  progression  changes  in  individuals  with  compromised 
immune  systems. 

Table  1 

.  Example  Learning  Objectives 

and  college  freshmen  levels,  and  w/ith  learning 
research  scientists. 

Development  of  valid  educational  content  cou¬ 
pled  with  simulations  of  biological  components 
and  their  processes  and  interactions  is  a  key 
component  of  the  project.  We  have  worked 
closely  with  leading  immunologists  and  the 
teachers  participating  in  the  evaluation  of  the 
game  to  develop  an  instructional  plan  for  the 
game  based  on  introductory  curricula,  including 
immunology  content,  determination  of  educa¬ 
tional  objectives,  strategy  for  integration  of 
learning  tools  into  the  game,  and  evaluation  of 
the  game.  Particular  emphasis  was  placed  on 
determination  of  appropriate  levels  of  content 
for  the  target  audience.  A  sample  of  the  learn¬ 
ing  objectives  is  shown  in  Table  1. 

A  major  challenge  of  the  project  was  to  define 
the  learning  content  so  that  it  would  be  easily 
accessible  and  understandable  both  by  the  bi¬ 
ologists  who  need  to  ensure  that  it  is  correct 
and  the  software  engineers  who  will  need  to 
instantiate  it  in  code.  One  of  the  great  chal¬ 
lenges  in  developing  reusable  software  objects 
is  that  they  are  often  so  difficult  to  understand 
that  it  is  easier  to  build  from  scratch  rather  than 
risk  misinterpreting  often  sparse  instructions 
about  how  to  use  legacy  objects.  We  have  de¬ 
veloped  formalisms  that  minimize  the  difference 
between  the  question  “how  does  this  object  op¬ 
erate?”  and  the  question  “how  does  the  biologi¬ 


cal  system  work?”  While  the  details  of  the  im¬ 
plementation  may  be  captured  in  complex  equa¬ 
tions,  the  vocabulary,  rules  represented,  and 
terms  used  to  describe  inputs  and  outputs  are 
all  easily  understandable  in  biological  terms. 

Our  formalism  is  based  on  the  Universal  Model¬ 
ing  Language  (UML),®  UML  tools  were  used  to 
build  a  formal  map  of  knowledge  about  the  im¬ 
mune  system  that  includes  named  variables 
and  formal  relationships  (that  should  be  covered 
by  the  ontology),  information  about  rules  and 
equations  describing  behavior,  clear  English 
descriptions  of  what  is  being  represented  avail¬ 
able  through  some  kind  of  introspection,  the 
referenced  source  of  the  information  encapsu¬ 
lated,  and  mechanisms  for  maintaining  version 
control.  The  structures  extend  the  structures  of 
the  ontology  to  ensure  consistent  inheritance  of 
both  fields  and  functions.  For  example,  a  com¬ 
ponent  representing  the  essential  elements  of  a 
cell  can  be  extended  to  represent  a  ge¬ 
neric  leukocyte  that  can  in  turn  be  extended  to 
represent  a  generic  lymphocyte  and  then  a  spe¬ 
cific  B  cell.  Such  structures  provide  a  seamless 
connection  between  ontologies  and  the  frame¬ 
work  of  software  objects.  It  provides  a  way  to 
ensure  that  functionality  and  updated  data  at 
the  root  of  an  object  tree  are  accurately  propa¬ 
gated. 

The  central  challenge  of  the  game  is  to  teach 
rules  to  a  set  of  players  that  represent  all  the 
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important  elements  of  the  immune  system  (e.g., 
macrophages,  neutrophils,  dendritic  cells,  T,  B, 
and  NK,  etc.)  These  rules  will  operate  the  soft¬ 
ware  objects  representing  the  cell  types.  As 
rules  are  turned  on,  more  complex  phenomena 
will  be  exposed.  A  key  part  of  the  educational 
game  is  understanding  the  way  the  characters 
(cell  types)  pass  messages  to  each  other  inter¬ 
act. 

The  levels  of  the  game  involve  an  ability  to  ad¬ 
dress  different  attack  strategies.  The  gameplay 
starts  with  a  simple  infection  that  can  be  mas¬ 
tered  with  the  innate  immune  system.  Progres¬ 
sively  more  sophisticated  viruses  and  bacteria 
require  mastery  of  more  sophisticated  strate¬ 
gies  and  understanding  how  to  use  an  increas¬ 
ing  number  of  immune  cells.  See  Table  2  for  a 
sample  of  the  levels.  The  core  challenge  and 
excitement  of  the  game  will  is  being  presented 
with  a  novel  class  of  infection  or  a  known  infec¬ 
tion  agent  at  a  new  site.  Each  will  simulate  the 
progression  of  an  actual  disease  -  which  can 
be  as  simple  as  a  simple  skin  infection  or  as 
complex  as  AIDS  or  TB.  At  an  early  stage,  for 
example,  a  player  may  be  confronted  by  a  sim¬ 
ple  skin  infection  that  can  be  addressed  by  us¬ 
ing  macrophages  and  the  innate  immune  re¬ 
sponse.  The  key  issue  is  telling  friend  from  foe. 
As  a  player  becomes  increasingly  comfortable 
with  play  at  this  level,  a  new  pathogen  is  intro¬ 
duced  that  multiplies  so  fast  that  the  macro¬ 
phages  are  overwhelmed.  A  new  strategy  must 


be  developed  to  send  signals  recruiting  help  - 
say  from  neutrophils.  This  means  understand¬ 
ing  the  signaling  and  the  combined  behaviors  of 
the  senders  and  receivers  of  the  signal.  As  the 
player  becomes  comfortable  managing  an  array 
of  diseases  at  this  level,  she  is  confronted  with 
infectious  agents  that  operate  from  within 
friendly  cells  such  as  viral  attacks.  This  re¬ 
quires  determining  a  strategy  for  detecting  the 
invasion  and  recruiting  specialized  help  from 
specialized  T  cells.  The  player  must  find  ways 
to  send  messengers  (e.g.,  dendritic  cells)  to 
lymph  nodes  that  may  be  distant  from  the  infec¬ 
tion.  Simulations  capable  of  supporting  this 
kind  of  game  play  must  be  able  to  operate  at 
many  different  levels  and  at  many  scales. 

At  each  level  the  student  can  elect  to  get  a 
briefing  on  the  material  or  elect  to  discover  the 
material  by  experimentation.  Help  is  available 
at  all  times  through  the  game’s  artificial  intelli¬ 
gence  and  from  teachers  and  peers. 

EVALUATION 

The  game  will  be  used  to  supplement  immunol¬ 
ogy  taught  as  a  part  of  biology  courses  given  to 
high  school  students  and  will  be  evaluated  in 
high  school  biology  classes.  The  project  will  be 
evaluated  based  on  the  degree  to  which  it  suc¬ 
cessfully  addresses  the  research  challenges 
identified  above.  These  challenges  center  on 
three  main  questions: 


^  '  Game  Play  Levels  (partial  list) 

LEVEL  1: 

Innate  Immunity-1 

The  student  will  be  able  to  analyze  the 
concept  of  how  immune  cells  recognize 
‘self  peptides  versus  ‘non-self  peptides 

The  challenge  is  to  train  macrophages  by  selecting  receptors  that 
identify  pathogens  and  then  choose  functions  that  help  to  destroy 

the  pathogens 

LEVEL2:  i 

Innate  ImmUiiity-2 

The  student  will  understand  how  cytokines 
and  other  proteins  that  can  recruit  neutro¬ 
phils  to  the  infection  site 

The  challenge  is  to  train  macrophages  to  release  the  appropriate 
signaling  proteins  and  train  neutrophils  in  the  blood  stream  to  rec¬ 
ognize  the  signal  and  behave  appropriately 

LEVEL  3: 

■  Adaptive-! 

The  student  will  understand  how  dendritic 
cells  become  active  and  carry  signals  to 
recruit  T8  cells  in  lymph  nodes 

To  succeed,  the  player  must  train  a  dendritic  cell  and  guide  iiiterac- 

tion  with  T  to  generate  clonal  expansion  bv  locating,  then  matching 
the  receptor  to  the  ligand 

LEVEL  4: 

Additional  levels  of  sophistication 

Table  2.  Game  Play  Levels 
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•  Has  the  project  developed  biologically  cor¬ 
rect,  visually  compelling  simulations  of  the 
immune  system  that  can  be  easily  navi¬ 
gated  by  people  without  specialized  techni¬ 
cal  proficiency? 

•  Do  the  simulations  effectively  incorporate 
state-of-the-art  open  architectures  and  in¬ 
teroperability  frameworks  that  will  both  fa¬ 
cilitate  revisions  and  augmentations  for  a 
variety  of  purposes  and  encourage  use  of 
simulation  components  in  the  work  of  oth¬ 
ers? 

•  Can  the  simulations  be  used  in  conjunction 
with  existing  instructional  tools  to  develop  a 
prototype  instructional  system  that  incorpo¬ 
rates  a  variety  of  assessment,  feedback, 
and  augmentation  tools? 

For  the  first  two  research  challenges,  assess¬ 
ments  will  be  conducted  through  workshops 
that  are  planned  annually  for  each  year  of  the 
project.  A  panel  of  experts  will  be  convened  as 
part  of  the  workshops  to  assess  the  biological 
accuracy  of  the  simulations  at  various  stages  of 
design  and  development  and  the  interoperability 
frameworks.  The  panel  will  consist  of  biologists, 
immunologists,  computational  biologists,  and 
medical  professionals.  The  evaluation  plan  for 
the  instructional  game  will  be  developed  with 
advice  by  the  Education  Advisory  Panel.  The 
evaluation  will  focus  on  four  key  questions: 

1. Does  use  of  the  instructional  game  improve 
the  performance  of  students  on  tests  now 
given  by  instructors  in  applicable  courses? 

2.  Does  the  system  improve  understanding  in 
areas  of  immunology  that  are  particularly 
difficult  to  master? 

3.  Does  the  system  increase  student  interest 
in  science  and  their  interest  in  a  career  in 
science? 

4.  Does  the  effect  of  1-3  depend  significantly 
on  sex,  ethnicity,  or  other  characteristics  of 
the  learners? 

NOVELTY/DISCUSSION 

Computer  games  hold  special  interest  to  a  gen¬ 
eration  who  has  grown  up  with  them,  and  as 
such,  they  show  promise  as  educational  tools. 
Whether  this  is  due  to  the  inherent  challenge 
built  into  game  play,  the  richness  of  graphics 
presented  to  the  user,  the  opportunity  to  interact 


with  other  users  (in  web-based  games),  the 
story  or  context  in  which  the  game  is  couched, 
or  some  other  feature  is  an  important  part  of 
this  research  project.  Exploiting  the  inherent 
motivational  aspects  of  games  and  simulations 
for  education  and  training  must  be  based  on  a 
sound  understanding  of  which  features  of  these 
systems  are  important  for  learning  and  why. 

The  instructional  game  will  be  used  to  supple¬ 
ment  immunology  taught  as  a  part  of  introduc¬ 
tory  biology  courses  given  to  high  school  stu¬ 
dents.  The  research  will  provide  the  basis  for 
building  engaging  simulations  that  teachers  can 
use  to  convey  subtle  concepts  in  the  immune 
system  and  infection  control.  It  should  improve 
biology  education  by  presenting  educational 
materials  in  a  way  that  is  engaging  and  useful 
to  people  with  a  variety  of  interests  and  back¬ 
grounds.  A  student  motivated  to  learn  the  com¬ 
plexities  of  the  immune  system  can  also  have 
the  basis  for  understanding  a  wide  range  of  bio¬ 
logical  concepts.  The  interest  generated  by  the 
game  could  encourage  students  from  many 
different  backgrounds  to  enter  a  scientific  or 
medical  profession.  While  the  initial  target  audi¬ 
ence  will  be  senior  high  school  and  first-year 
college  students,  we  expect  that  the  system  will 
facilitate  learning  for  a  broad  class  of  learners, 
including  the  many  professionals  that  need  to 
master  elementary  concepts  in  infection  and 
infection  control  -  such  as  people  who  now 
must  be  trained  to  deal  with  potential  biological 
attacks.  The  research  will  result  in  tools  that  can 
assist  people  to  improve  their  own  health  care 
by  presenting  difficult  concepts  in  ways  that  a 
typical  patient  can  understand.  It  is  also  likely 
to  help  research  specialists  get  a  clear,  visual 
understanding  of  the  way  their  research  special¬ 
ties  fit  into  the  extravagantly  complex  network  of 
operations  that  combine  to  make  the  immune 
system. 
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Abstract:  Geometry  education  has  proven  as  one  powerful  means  of  improving  spatial  abilities,  an 
important  component  of  human  intelligence.  In  the  first  part  of  this  paper  we  summarize  our  develop¬ 
ment  of  a  system  that  uses  collaborative  augmented  reality  as  a  medium  for  teaching,  and  uses  3D 
dynamic  geometry  to  facilitate  mathematics  and  geometry  education.  Our  immersive  collaborative 
educational  application,  specifically  developed  for  geometry  education,  serves  as  the  basis  of  a  com¬ 
prehensive  evaluation  study  regarding  its  efficacy  in  training  spatial  abilities.  The  main  contribution  is 
the  description  of  evaluation  design  including  the  test  instruments,  learning  tasks  and  practical  ex¬ 
periences  with  using  our  system  for  actual  training  of  high  school  students.  Results  of  a  pre-study 
with  spatial  ability  tests  in  high  schools  are  presented.  They  point  to  interesting  gender-specific  differ¬ 
ences  of  strategies  when  solving  spatial  ability  tests,  which  have  not  been  reported  in  literature  be¬ 
fore. 


INTRODUCTION 

Spatial  abilities  present  an  important  component 
of  human  intelligence.  Many  studies  have 
shown  that  spatial  abilities  can  be  improved  by 
well-designed  trainings.®^  Geometry  education 
has  proven  as  one  powerful  means  of  improving 
these  skills; recently,  a  number  of  training 
studies  have  shown  the  usefulness  of  virtual 
reality  (VR)  in  training  spatial  ability.®'^®  How¬ 
ever,  little  to  no  work  has  been  done  towards 
systematic  development  of  VR  applications  for 
practical  education  purposes  in  this  field. 

•  No  VR/AR  application  for  actual  use  in  high 
school  or  higher  education  has  ever  been 
developed  with  the  main  purpose  of  improv¬ 
ing  spatial  skills. 

•  Hardly  any  evaluations  can  be  found  in  lit¬ 
erature  which  give  hints  to  the  actual  learn¬ 
ing  transfer  from  a  VR/AR  learning  environ¬ 
ment  to  the  real  world. 

The  authors  initiated  a  national  research  project 
which  addresses  these  two  key  issues  by  devel¬ 
oping  a  prototype  AR  system  for  geometry  edu¬ 
cation.  As  a  pilot  study  for  this  proposal,  a  three 
dimensional  geometric  construction  tool  called 
ConstructSD^®'^®  has  been  developed  that 
serves  as  a  platform  for  our  work.  A  compre¬ 


hensive  evaluation  study  will  be  conducted  in 
order  to  study  the  general  and  differential  ef¬ 
fects  of  the  training  on  several  components  of 
spatial  ability. 

In  this  paper,  we  briefly  describe  our  experi¬ 
ences  in  developing  and  using  ConstructSD  in 
high  school  geometry  education.  To  provide  a 
natural  face-to-face  setting  for  teachers  and 
students,  this  system  uses  an  immersive  setup, 
more  specifically  a  collaborative  augmented 
reality  (AR)  setup,  based  on  see-through  head- 
mounted  displays  (HMD).  The  main  advantage 
of  using  AR  is  that  students  actually  see  three 
dimensional  objects  which  they  until  now  had  to 
calculate  and  construct  with  traditional  -  mostly 
pen  and  paper  -  methods  (Figure  1).  The  face- 
to-face  setting  allows  for  traditional  pedagogic 
communication. 

It  is  important  to  note  that  while  geometry  edu¬ 
cation  software  shares  many  aspects  with  con¬ 
ventional  computer  aided  design  software  at  a 
first  glance,  its  aims  and  goals  are  fundamen¬ 
tally  different.  Geometry  education  software  is 
not  intended  for  generating  polished  results,  but 
puts  an  emphasis  on  the  construction  process 
itself.  While  relatively  simple  geometric  primi¬ 
tives  and  operations  will  suffice  for  the  intended 
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Figure  1.  Students  are  working  in  3D  space 
with  ConstructSD. 

audience  of  age  10  to  20  (especially  10  to  14), 
the  user  interface  must  be  both  intuitive  and 
instructive  in  terms  of  the  provided  visualiza¬ 
tions  and  tools.  Commercial  CAD  software  of¬ 
fers  an  overwhelming  variety  of  complex  fea¬ 
tures  and  often  has  a  steep  learning  curve.  In 
contrast,  geometry  educators  are  interested  in 
simple  construction  tools  that  expose  the  under¬ 
lying  spatial  process  in  a  comprehensive  way. 
The  interface  might  be  restricted  to  their  stu¬ 
dents  needs.  In  accordance  to  that  our  aim  with 
ConstructSD  was  not  to  create  a  professional 
3D  modeling  package  but  a  simple  and  intuitive 
3D  construction  (not  animation)  tool  in  an  im¬ 
mersive  environment  for  educational  purposes. 
The  main  part  of  our  work  concentrates  on  our 
evaluation  study  -  the  test  instruments,  partici¬ 
pants  and  the  study  design.  We  intend  to  ad¬ 
dress  the  following  research  questions  in  our 
evaluation  study. 

1.  Effect  of  the  training  on  performance  in 
tasks  central  to  the  training; 


2.  Transfer  of  the  training  effect  to  more  dis¬ 
tant  spatial  tasks; 

3.  Effect  of  the  training  on  strategy  use  in  cen¬ 
tral  and  distant  spatial  tasks; 

4.  Aptitude-treatment  interactions:  Depend¬ 
ence  of  individual  training  effects  on  pretest 
spatial  ability,  verbal  ability,  and  reasoning 
ability; 

5.  Gender  differences  in  training  effects. 

RELATED  WORK 

For  the  development  of  any  educational  VR/AR 
application,  technological,  domain  specific, 
pedagogical  and  psychological  aspects  are  of 
importance.  Accordingly,  literature  from  different 
and  diverse  research  areas  relates  to  our  work: 
VR/AR  collaborative  distributed  systems,  desk¬ 
top  3D  modeling,  immersive  modeling,  educa¬ 
tional  2D/3D  applications,  dynamic  2D  geome¬ 
try,  parametric  CAD,  pedagogic  theories  such 
as  constructivism  or  activity  theory  and  psycho¬ 
logical  literature  from  the  field  of  spatial  ability 
research.  We  will  briefly  mention  work  related  to 
training  and  psychological  aspects  of  this  paper. 
For  a  comprehensive  overview  of  related  work 
we  refer  to.^® 

Educational  VR  Applications 

Since  the  early  1990th  researchers  have  been 
working  on  virtual  reality  applications  for  purely 
educational  use.^  ®  In  the  area  of  mathematics 
education  the  most  recent  and  most  advanced 
project  is  CyberMath.®"*  CyberMath  is  an  avatar- 
based  shared  virtual  environment  aimed  at  im¬ 
proving  mathematics  education.  It  is  suitable  for 
exploring  and  teaching  mathematics  in  situa¬ 
tions  where  both  the  teacher  and  the  students 
are  co-present  and  physically  separated.  A  very 
good  summary  of  educational  and  training  appli¬ 
cations  is  given  by  Mantovani.^® 

Training  Spatial  Abilities  in  VR 

Training  of  spatial  abilities  in  VR  is  of  strong 
interest  in  application  areas  such  as  surgery,® 
navigation  and  way  finding,®  for  rehabilitation  of 
patients  with  psychiatric  illnesses  such  as  trau¬ 
matic  stress  disorder,  schizophrenia,  and  Alz¬ 
heimer's  disease^®'^®  or  for  pre-flight  space  train¬ 
ing. 
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A  recent  article  by  Durlach  et  al.®  gives  a  very 
good  overview  of  work  that  has  already  been 
done  in  the  area  of  enhancing  spatial  skills 
within  virtual  environments  but  mainly  identifies 
the  indispensable  need  for  comprehensive  fu¬ 
ture  research  in  this  area. 

Spatial  Cognition 

Psychological  literature  on  spatial  cognition 
ranges  from  studies  on  the  way  people  process 
simple  spatial  objects  to  studies  about  how  peo¬ 
ple  orient  themselves  in  real  environments.  In 
the  following,  we  give  a  brief  introduction  into 
spatial  cognition  with  references  to  related  work. 
Several  authors®  ''®'^®'®®  suggested  how  spatial 
ability  can  be  structured  into  sub  domains.  Most 
of  the  proposed  structures  focus  on  relation¬ 
ships  and  similarities  among  spatial  tests  and 
were  developed  through  analysis  of  test  inter¬ 
correlations.  The  results  of  structural  analyses 
of  spatial  ability  are  highly  convergent.  In  addi¬ 
tion  to  some  simple,  basic  performance  as¬ 
pects,  at  least  two  factors  are  consistently  re¬ 
ported:  “Spatial  Relations”  (as  labeled  by  Leh¬ 
man^®),  that  is,  speeded  mental  rotation,  and 
“Visualization”,  which  includes  all  complex, 
multi-step  spatial  tasks.  Tasks  involving  three- 
dimensional  mental  rotation  are  somewhat  inter¬ 
mediate  and  have  been  grouped  into  each  of 
these  two  factors.  Tasks  requiring  participants 
to  imagine  different  perspectives  either  form  a 
third  factor  or  are  grouped  into  Spatial  Rela¬ 
tions. 

A  different  approach  to  thinking  about  spatial 
cognition  is  by  analyzing  the  processes  people 
actually  use  to  solve  spatial  tasks.  Basically,  all 
spatial  tasks  can  be  solved  in  different  ways, 
and  the  more  complex  a  task  is,  the  more  differ¬ 
ent  strategies  can  be  used  to  solve  it  (overview 
in®'*).  People  differ  in  the  strategies  they  use, 
and  people  shift  their  strategies  within  a  task  if 
necessary.  A  basic  distinction  is  between  holis¬ 
tic  and  analytic  strategies.  Holistic  strategies 
involve  representing  and  manipulating  spatial 
information  “in  a  spatial  way”,  that  is,  mentally 
using  information  about  spatial  relations  be¬ 
tween  elements  in  the  mental  representation.  A 
person  using  holistic  strategies  imagines,  for 
example,  how  a  stimulus  figure  is  rotated,  or 
how  a  two-dimensional  shape  can  be  folded  into 
a  three-dimensional  object.  Analytic  strategies 


reduce  spatial  information  to  an  essentially  non- 
spatial,  serial  format.  For  example,  a  route  can 
be  represented  as  a  list  of  landmarks,  and  the 
spatial  relations  among  the  patterns  on  a  cube 
can  be  represented  as  a  list  of  relations  among 
pairs  of  patterns.  Thus,  the  complexity  of  spatial 
information  is  reduced,  and  the  focus  is  on  parts 
of  the  object  or  the  environment  rather  than  on 
the  object  as  a  whole.  Compared  to  holistic 
strategies,  analytic  strategies  usually  take  more 
time,  but  less  mental  effort,  as  the  information 
represented  is  less  complex.  Note  that  analytic 
and  holistic  strategies  should  not  be  viewed  as 
mutually  exclusive  categories. 

Gender  differences  in  strategy  use  have  fre¬ 
quently  been  found  in  studies  of  environmental 
orientation  and  navigation.  Men  more  often  ac¬ 
quire  map-like  knowledge  about  their  environ¬ 
ment,  use  Euclidean  information,  and  are  aware 
of  relative  directions,  whereas  women  more  of¬ 
ten  rely  on  landmarks  and  represent  routes  as 
lists  of  landmarks.  These  gender  differences 
have  been  found  in  self-reported  strategy  use  in 
new  environments®^  and  in  use  of  landmarks 
and  Euclidean  information  in  learning  routes 
from  maps  and  giving  directions.'' With  respect 
to  spatial  tests,  gender  differences  in  strategy 
use  have  not  been  studied  much,  but  at  least 
one^’  found  that  men  more  often  used  holistic 
strategies  than  women,  and  women  more  often 
used  analytic  and  mixed  strategies  than  men  in 
two  different  spatial  tests. 


Figure  2.  Two  students  collaborate  in  our  stan¬ 
dard  lab  setup,  wearing  an  HMD,  holding  a 
wireless  pen  and  panel.  All  devices  are  optically 
tracked. 
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CONSTRUCT3D 

ConstructSD  is  based  on  the  Studierstube  AR 
system^°  and  uses  augmented  reality  to  provide 
a  natural  setting  for  face-to-face  collaboration  of 
teachers  and  students. 

Hardware  Setup 

In  our  standard  Studierstube  setup,  we  have  2 
collaborating  users  wearing  HMDs  for  a  shared 
virtual  space,  and  holding  fuily  tracked  interac¬ 
tion  props  in  their  hands.  One  dedicated  host 
with  2  graphic  ports  renders  stereoscopic  views 
for  both  users.  We  are  using  this  setup  for  dem¬ 
onstrations  and  evaluations.  Other  setups  for 
educational  use  have  been  reported  in.^® 

Software  Design 

The  current  version  of  ConstructSD  offers  func¬ 
tions  for  the  construction  of  points,  two- 
dimensional  geometric  primitives  and  three- 
dimensional  geometric  objects.  It  provides  func¬ 
tionality  for  planar  and  spatial  geometric  opera¬ 
tions  on  these  objects,  allows  measurements, 
features  structuring  of  elements  into  layers  and 
offers  basic  system  functions. 

ConstructSD  promotes  and  supports  exploratory 
behavior  through  dynamic  geometry.  A  funda¬ 
mental  property  of  dynamic  geometry  software 
is  that  dynamic  behavior  of  a  construction  can 
be  explored  by  interactively  moving  individual 


defining  elements  such  as  corner  points  of  a 
rigid  body.  It  can  be  seen  what  parts  of  a  con¬ 
struction  change  and  which  remain  the  same. 
The  histories  of  constructions  as  well  as  de¬ 
pendencies  between  geometric  objects  are 
maintained.  Experiencing  what  happens  under 
movement  allows  better  insight  into  a  particular 
construction  and  geometry  in  general. 

At  its  start  ConstructSD  initializes  a  3D  window 
which  has  maximum  size  to  cover  the  “whole” 
virtual  space.  The  user  interface  is  initialized, 
the  menu  system  is  mapped  to  a  hand-held 
tracked  panel  called  the  personal  interaction 
panel  (PIP).  The  PIP  allows  the  straightforward 
integration  of  conventional  2D  interface  ele¬ 
ments  like  buttons,  sliders,  dials  etc.  as  well  as 
novel  3D  interaction  widgets.  Haptic  feedback 
from  the  physical  props  guides  the  user  when 
interacting  with  the  PIP,  while  the  overlaid 
graphics  allows  the  props  to  be  used  as  multi¬ 
functional  tools. 

All  construction  steps  are  carried  out  via  direct 
manipuiation  in  3D  using  a  stylus  tracked  with 
six  degrees  of  freedom.  In  order  to  generate  a 
new  point  the  user  clicks  with  his  pen  exactly  at 
the  location  in  3D  space  where  the  point  should 
appear.  Users  can  easily  switch  between  point 
mode  (for  setting  new  points)  and  selection 
mode  (for  selecting  3D  objects).  We  omit  a  de¬ 
tailed  description  of  the  user  interface  in  this 
context. 


Figure  3.  Left:  Surface  of  revolution  with  the  tangential  plane  in  a  surface  point.  Right:  Two  students 
are  collaborating  to  solve  the  problem.  Vienna  is  marked  on  the  globe  and  a  geostationary  satellite 

(in  the  equatorial  plane)  is  given  too. 
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All  operations  consistently  support  dynamic 
modifications  of  their  input  elements  and  re¬ 
evaluate  the  resulting  elements  accordingly. 

LEARNING  TASKS 


images  help  to  understand  and  translate  the 
problem.  The  given  virtual  scene  in  ConstructSD 
(Figure  3  right)  shows  a  model  of  earth  (with 
texture)  to  help  pupils  find  the  correct  places  on 
earth  and  to  immerse  them  further  into  the  prob¬ 
lem. 


To  illustrate  the  type  of  geometric  examples  and 
learning  tasks  that  are  best  suited  for  teaching 
geometry  with  ConstructSD  we  list  actual  edu¬ 
cational  tasks  and  examples  that  we  used  in 
previous  evaluations.  Students  of  age  16-18 
who  have  geometry  classes  in  high  school 
worked  collaboratively  with  a  teacher  and  an¬ 
other  student  on  these  examples. 

Surface  of  Revolution 

Given  an  axis,  students  have  to  construct  a  sur¬ 
face  of  revolution  by  rotating  a  B-Spline  curve 
(cubic,  5-6  control  points)  around  the  axis.  The 
control  points  can  dynamically  be  modified  at 
any  time  resulting  in  a  change  of  the  surface  of 
revolution. 


Deflection  Sheave 

A  rope  is  being  redirected  from  one  given  posi¬ 
tion  to  a  final  given  position.  Two  deflection 
sheaves,  which  are  drawn  as  circles  redirect  the 
rope.  They  have  to  be  constructed  as  well  as 
the  piece  of  connecting  rope  between  them. 
Only  start  and  end  position  of  the  rope  are  given 
as  well  as  the  mid  point  of  the  first  deflection 
sheave.  Deflection  sheaves  can  be  found  at 
skiing  lifts,  elevators  and  in  many  other  ma¬ 
chines.  Students  got  an  explanatory  draft  to 
lead  them  towards  a  correct  solution. 

USABILITY  EVALUATIONS 

System  Evaluation 


In  the  next  step,  the  tangential  plane  in  an  arbi¬ 
trary  point  P  of  the  surface  must  be  found. 
Therefore  pupils  have  to  construct  a  meridian 
curve  through  P  which  they  get  by  intersecting 
the  surface  with  a  plane  through  the  axis.  They 
also  have  to  rotate  the  point  around  the  axis  to 
get  its  circle  of  latitude  on  the  surface  of  revolu¬ 
tion.  The  two  tangents  to  the  circle  of  latitude 
and  to  the  meridian  curve  in  P  define  the  tan¬ 
gential  plane.  Figure  3  (left)  shows  the  result  of 
this  construction. 

Dynamic  modifications  of  the  start  elements  and 
all  constructed  parts  allow  to  study  and  explore 
the  construction.  Students  see  immediately  and 
learn  that  the  surface  normal  in  any  point  of  the 
surface  intersects  the  surface’s  axis,  which  is  an 
important  property  of  surfaces  of  revolution. 

Adjusting  a  Satellite  Dish 

In  the  second  problem  a  satellite  dish,  posi¬ 
tioned  in  Vienna,  has  to  be  adjusted  to  point  to 
the  TV-SAT2  satellite,  which  is  given.  Students 
have  to  translate  this  real  life  problem  into  a 
geometric  problem  to  be  able  to  identify  two 
angles  which  are  needed  to  adjust  the  satellite 
dish.  Web  links  are  presented  with  additional 
information  about  geostationary  satellites;  given 


Based  on  feedback  from  many  trials  with  real 
high  school  students,  we  continuously  improved 
ConstructSD  over  a  period  of  3  years.  In  early 
2004,  we  conducted  a  larger  study  based  on 
interviews  and  the  standardized  ISONORM 
9241/10  usability  questionnaire^®  after  system¬ 
atic  exposure  of  15  students  (9  male,  6  female) 
to  ConstructSD.  A  number  of  training  exercises 
that  fit  the  students'  11*^  and  12*’^  grade  curricu- 


ISONORM  Usability  Questionnaire 


-3.00  -2.00 


-1,00 


0,00  1,00  2,00  3,00 


□  Suitability  for  the  Task 

■  Self-Descriptiveness 

□  Controllability 

□  Confcrmily  with  User 
Expectations 

■  Error  Tolerance 

B  Suitability  for  Learning 


Table  1.  summarizes  the  results  of  the 
ISONORM  usability  questionnaire  in  the  6  cate¬ 
gories  “suitability  for  the  task,”  “self¬ 
descriptiveness,”  “controllability,”  “conformity 
with  user  expectation,”  “error  tolerance,”  and 
“suitability  for  learning”. 
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lum  were  designed  by  us  and  then  worked 
through  by  students  from  two  high  schools  with 
the  aid  of  their  teachers.  All  students  attended 
to  geometry  classes  (descriptive  geometry) 
since  the  beginning  of  grade  1 1 .  Each  of  them 
participated  in  5  training  sessions  lasting  6 
hours.  Our  main  objective  was  to  at  least  infor¬ 
mally  assess  the  usability  and  potential  of  our 
system  and  method  for  real  high  school  work. 

ConstructSD  Usability  Results 

At  the  end  of  all  training  sessions  students  had 
to  answer  an  ISONORM  usability  questionnaire. 
Two  questions  regarding  self-descriptiveness  of 
the  application  had  to  be  removed  since  they 
were  related  to  desktop  applications  only.  After¬ 
wards  students  answered  general  questions 
regarding  user  acceptance,  user  behaviour, 
technical  requirements  and  organisational  as¬ 
pects. 

A  closer  look  at  the  data  reveals  that  the  cate¬ 
gories  “suitability  for  learning”  and  “suitability  for 
task”  received  the  highest  grading  which  is  very 
important  in  this  context.  In  our  opinion  the  high¬ 
est  priorities  for  an  educational  application  that 
complies  with  pedagogic  theories  such  as  con¬ 
structivism  are  that  it  (1)  is  easy  to  use  and  re¬ 
quires  little  time  to  learn,  (2)  encourages  learn¬ 
ers  to  try  new  functions  and  (3)  can  be  used 
consistently  and  is  designed  in  a  way  that  things 
you  learned  once  are  memorized  well.  These 
are  exactly  the  items  that  students  rated  very 
high.  Nearly  all  students  reported  that  they  can 
imagine  using  the  current  version  of  Con- 
structSD  in  high  school  or  university  education. 
The  categories  “self-descriptiveness”  and 
“conformity  with  user  expectations”  got  lower 
grades  than  the  rest.  The  self-descriptiveness  of 
ConstructSD  is  currently  improved  by  adding 
better  labeling  and  tool  tips  to  widgets  on  our 
panel  in  order  to  explain  all  menu  items.  We  are 
also  restructuring  the  menu  system  to  give  a 
better  overview  of  the  functionality. 

Some  of  the  students  reported  negative  side 
effects  after  working  in  the  virtual  environment. 
One  female  student  reported  headache  and 
eyestrain  after  20  minutes  of  work  in  the  virtual 
environment  but  did  not  stop  working  and 
wanted  to  use  ConstructSD  again  (in  total  she 
worked  for  3  hours  with  the  system).  In  retro¬ 
spect  we  know  that  our  one  hour  lessons  were 


simply  too  long  for  continuous  work  with  an 
HMD.  Since  negative  side  effects  are  a  general 
problem  when  working  with  HMDs  and  influence 
the  user’s  subjective  experience  of  a  VR/AR 
environment  considerably  they  are  relevant  to 
all  VR/AR  applications  that  use  HMDs.  We  iden¬ 
tified  some  possible  reasons  of  cybersickness 
that  may  be  relevant  to  our  virtual  environment 
such  as  accommodation  problems,  low  frame 
rate,  lag  or  bad  fitting  helmets.  In  the  meantime 
we  already  reduced  some  of  these  negative 
side  effects  (using  light  weight  bicycle  helmets 
and  increasing  frame  rate). 

We  must  stress  that  we  designed  Construct3D 
to  be  a  valuable  addition  to  traditional  or  com¬ 
puter  aided  geometry  education  and  not  a  sub¬ 
stitute  of  it.  A  very  reasonable  time  for  working 
and  teaching  with  ConstructSD  during  a  class  is 
about  20-30  minutes.  According  to  our  observa¬ 
tions,  this  would  guarantee  that  less  than  5%  of 
all  students  feel  any  negative  side  effects  such 
as  eyestrain  or  even  headache.  Of  course  it 
would  be  desirable  to  find  the  exact  reasons  of 
cybersickness  and  ways  to  avoid  it  -  as  pointed 
out  by  other  researchers. 

EVALUATION  DESIGN  ON  TRAINING  SPA¬ 
TIAL  ABILITIES 

The  presented  system  serves  as  a  base  of  fur¬ 
ther  research  and  allows  to  do  more  in  depth 
evaluations  of  various  aspects  regarding  VR/AR 
learning  that  have  not  been  studied  before. 

Currently  we  focus  on  the  largest  evaluation 
with  Construct3D  yet,  involving  over  250  stu¬ 
dents.  It  concentrates  on  5  main  questions 
about  training  and  improving  spatial  abilities  as 
mentioned  in  the  introduction. 

Test  Groups 

We  will  conduct  a  pre-/post-test  experiment  with 
four  different  training  groups  and  one  untrained 
control  group.  At  pre-test,  all  participants  will  be 
presented  with  a  battery  of  spatial  tests 
(including  strategy  assessments)  and  tests  of 
verbal  and  reasoning  ability.  Then,  each  partici¬ 
pant  will  be  randomly  assigned  to  one  of  five 
groups: 

a.  Untrained  control  group:  Participates  in  the 
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pre-  and  post-test  but  does  not  receive  any 
training; 

b.  Traditional  school  group;  Participates  in  the 
pre-  and  post-test,  but  is  taught  geometry  in 
their  normal  school  classes  between  pre- 
and  post-test  with  traditional  paper  &  pencil 
methods; 

c.  Modern  instruction/school  group:  Partici¬ 
pates  in  the  pre-  and  post-test,  but  is  taught 
geometry  in  their  normal  school  classes 
between  pre-  and  post-test  on  the  computer 
using  CAD  software; 

d.  Individual  tutoring  group;  works  with  a  tutor 
using  standard  teaching  methods  without 
ConstructSD; 

e.  ConstructSD  group:  works  with  Con- 
structSD. 

f. 

The  reason  we  use  an  individual  tutoring  group 
is  that  this  condition  is  probably  “harder  to  beat” 
than  standard  class-wise  Descriptive  Geometry 
(DG)  instruction.  Tutors  are  likely  to  use  hand 
drawing  and  sketching  as  teaching  methods, 
thus,  they  will  provide  a  “hands-on”  experience 
that  has  proven  to  be  particularly  effective  in 
other  training  studies.  However,  we  expect  the 
ConstructSD  tutorial  to  lead  to  broader  transfer 
of  training  gains  than  the  tutoring.  As  women 
more  often  than  men  use  analytic  strategies  on 
spatial  tests,  the  ConstructSD  training,  which 
focuses  on  visualization,  might  have  different 
effects  on  female  than  on  male  participants.  It  is 
certainly  interesting  to  see  how  different  types 
of  training  affect  such  differences. 

Between  pre-  and  post-tests  five  training  ses¬ 
sions  will  be  conducted.  Participants  will  be  stu¬ 
dents  in  their  11th  and  12th  year  of  schooling, 
attending  secondary  schools  in  Vienna.  Austria 
has  several  different  secondary-education  cur¬ 
ricula,  some  of  which  have  a  focus  on  the  natu¬ 
ral  sciences,  including  Descriptive  Geometry 
(DG).  DG  classes  start  in  the  1 1th  form. 

Instruments 

The  following  instruments  will  be  presented  to 
participants  at  pre-  and  post-test.  A  battery  of 
spatial  tests  as  listed  below  will  be  used: 

•  Mental  Rotations  Test  (MRT^®),  German 
version  by:^^  speeded  mental  rotation. 

•  Purdue  Spatial  Visualization  Test:  Rotations 


(PSVT:R^^):  shortform:  mental  rotation  com¬ 
bined  with  visualization. 

•  Mental  Cutting  Test  (MCT^):  shortform:  visu¬ 
alization. 

•  Differential  Aptitude  Test:  Space  Relations 
(DAT:SR^^):  shortform:  visualization. 

•  Spatial  Orientation  Test  (SOT^®);  mental 
rotation  -  large  scale 

•  Judgement  of  Line  Orientation  Test  (JLO®); 
visuospatial  perception  and  processing. 

Reasoning  and  Verbal  Ability  Tests: 

•  “Letter  Sequence”:  subtest  from  the  Ger¬ 
man  intelligence  test  battery  “Wilde  Intelli- 
genz  Test”  (WIT)  by:^'*  shortform 

•  “Verbal  analogies”:  subtest  from  the  Ger¬ 
man  intelligence  test  battery  “Berliner  Intelli- 
genzstruktur  Test”  (BIS)^® 

•  "Vocabulary”  ("Wortschatztest"  WST):®^ 
shortform 

Strategy  Questionnaire:  Assessment  of  different 
aspects  of  holistic  vs.  analytic  strategies,  the 
most  important  ones  are  “Visualization  /  Imagi¬ 
nation”  vs.  “Non-visual  Reasoning  /  Thinking” 
and  “Focus  on  the  object  as  a  whole”  vs.  “Focus 
on  details”. 

These  tests  have  also  been  used  at  two  prelimi¬ 
nary  studies  that  we  did  in  Austrian  high 
schools.  In  the  first  study  42  students  (39 
males,  3  females;  aged  17-19  years,  11*^  grade) 
participated,  in  the  second  bigger  one  there 
were  79  students  (44  females,  33  males,  2  un¬ 
known;  aged  16-19,  1l“^  and  12*^  grade).  Due  to 
time  restrictions  in  schools,  we  developed  two 
short  versions  of  each  test  except  the  MRT 
which  was  used  in  its  full  length.  We  split  the 
tests  (PSVT:R,  MCT,  DAT:SR)  into  halves  by 
simply  assigning  all  odd  items  to  test  version  A 
and  all  even  ones  to  test  version  B.  In  order  to 
eliminate  obvious  inequalities,  we  thereafter 
exchanged  some  items  or  assigned  some  items 
to  both  test  versions.  The  final  test  forms  con¬ 
sisted  of  15  (MCT  and  PSVT:R)  respectively  25 
(DAT:SR)  items.  The  time  limits  set  for  the  tests 
were  calculated  based  on  the  regular  test  ver¬ 
sions  and  the  number  of  problems  selected  for 
the  short  versions  and  were  therefore  12.5  min¬ 
utes  for  the  PSVT:R,  12  minutes  for  the  MCT 
and  12.5  minutes  for  the  DAT:SR.  Our  short  test 
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versions  had  reliability  coefficients  (Cronbach’s 
alphas)  ranging  form  0.66  to  0.90. 

PRELIMINARY  STUDY 

The  purpose  of  the  preliminary  study  was  to 
examine  some  frequently  employed  spatial  tests 
that  contain  different  kinds  of  tasks  and  are  sup¬ 
posed  to  affect  different  aspects  of  spatial  ability 
in  regard  to  performance,  strategy  use  and  the 
relation  between  these  two  variables.  More  pre¬ 
cisely,  we  aimed  to  find  answers  to  the  following 
questions: 

a.  How  do  performances  in  different  spatial 
ability  tests  correlate  with  each  other  and 
what  is  the  nature  of  the  relationship  to  cog¬ 
nitive  tests  measuring  analytical  abilities 
(reasoning  and  verbal  abilities)? 

b. Are  there  intra-individual  differences  in 
strategy  use  across  tests?  Do  different  spa¬ 
tial  tests  evoke  different  strategies? 

c. Are  there  /nfer-individual  differences  in  the 
use  of  strategies?  Do  participants  show  a 
tendency  to  use  holistic  or  analytic  strate¬ 
gies  irrespective  of  the  nature  of  the  task? 

d. How  are  performance  and  strategy  use  as¬ 
sociated  with  each  other? 

RESULTS 

The  first  study  (39  males,  3  females;  aged  17- 
19  years,  1 1®’  grade)  aimed  to  examine  associa¬ 
tions  and  intra-  and  inter-individual  differences 
between  several  spatial  ability  tests  regarding 
performance  and  strategy  use  and  to  make  con¬ 
nections  between  performance  and  strategy 
use.  The  spatial  ability  tests  used  were  the 
MRT,  the  PSVTiR,  the  DAT:SR  and  the  MCT 
together  with  reasoning  (verbal  and  non-verbal) 
and  verbal  ability  (vocabulary)  tests. 

In  an  explorative  pre-study  subjects  were  asked 
to  solve  problems  of  these  tests  by  thinking 
aloud.  The  findings  of  this  pre-study  suggest 
that  a  major  part  of  these  problems  (especially 
PSVT:R  and  DAT:SR  problems)  can  be  suc¬ 
cessfully  solved  not  only  in  a  spatial  (holistic) 
way  but  also  by  using  analytic  non-spatial 
strategies.  The  way  the  MCT  was  solved  indi¬ 
cates  that  (geometry)  knowledge  may  also  be 
an  important  factor. 

Applying  a  factor  analysis  on  the  test  scores  3 


factors  could  be  identified:  spatial  ability  (with 
loadings  of  the  4  spatial  ability  tests  as  well  as  a 
lower  loading  of  non-verbal  reasoning  tests), 
reasoning  ability  (with  loadings  of  both  reason¬ 
ing  tests),  and  vocabulary.  There  were  positive 
significant  correlations  between  the  DAT:SR 
and  the  PSVT:R  on  one  hand  and  the  non¬ 
verbal  reasoning  test  on  the  other  hand.  This 
finding  goes  together  with  the  one  from  the  pre¬ 
study  indicating  that  DAT:SR  and  PSVT:R 
scores  are  not  a  pure  measure  of  “real”  spatial 
(holistic  /  visualization)  ability  but  confounded  by 
reasoning  abilities. 

Analyses  of  the  strategy  use  questions  reveal 
that,  on  the  one  hand,  there  are  differences  in 
strategy  use  on  an  inter-individual  level:  Signifi¬ 
cant  positive  correlations  over  the  tests  indicate 
that  subjects  tend  to  certain  strategies  that  are 
rather  independent  from  the  kind  of  spatial  task. 
On  the  other  hand,  these  personal  strategy  ten¬ 
dencies  are  not  completely  unaffected  by  the 
nature  of  the  spatial  test:  The  use  of  holistic 
strategies  is  significantly  higher  for  the  MRT  as 
for  the  other  three  spatial  tests. 

When  drawing  the  conjunction  between  per¬ 
formance  and  strategy  use,  we  found  that  there 
were  no  correlation  between  performance  and 
strategy  regarding  the  DAT:SR,  the  PSVT:R, 
and  the  MCT,  i.e.,  subjects  using  mainly  holistic 
strategies  could  not  achieve  higher  scores  than 
subjects  using  mainly  analytic  scores.  However, 
there  was  a  significant  correlation  between 
strategy  use  and  performance  in  the  MRT,  indi¬ 
cating  that  “holistic"  subjects  scored  slightly  bet¬ 
ter  than  “analytic”  subjects. 

Considering  these  results,  it  seems  that  the 
MRT  stands  out  from  the  other  three  spatial 
tests  we  used  in  this  study:  It  is  processed  more 
holistic  than  the  other  tests  and  the  use  of  holis¬ 
tic  strategies  has  more  (positive)  influence  on 
the  performance  compared  to  the  DAT:SR,  the 
PSVT:R  and  the  MCT. 

Altogether,  our  findings  give  some  evidence  that 
spatial  ability  tests  -  in  particular  the  PSVT:R 
and  the  DAT:SR  -  can  be  successfully  solved 
not  only  with  “real”  spatial  (holistic)  strategies 
but  also  with  analytical  strategies  -  at  least  as 
long  the  time  limit  set  is  ample  enough  as  these 
analytical  strategies  are  generally  more  time- 
consuming  than  holistic  processing.  Results 
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Component 

1 

2 

PS\/T:R  score 

,689 

,363 

MCT  score 

,761 

,269 

MRT  score 

,094 

,890 

DAT  score 

,707 

,481 

WIT  score 

,736 

,392 

JLO  score 

,211 

,682 

Component 

1 

2 

PSVT'.R  score 

,661 

,196 

MCT  score 

,735 

-,141 

MRT  score 

,632 

,231 

DAT  score 

,836 

,083 

WIT  score 

,187 

,888 

JLO  score 

,826 

-,056 

Table  2.  Component  matrix  for  male  partici¬ 
pants 

from  an  expiorative  pre-study  also  suggest,  that 
knowledge  plays  an  important  role  in  the  solving 
process  of  the  MCT.  Though  our  results  are 
based  on  a  rather  small  sample  it  seems  to  be 
justifiable  to  draw  the  tentative  conclusion  that 
the  scores  of  spatial  ability  tests  do  not  neces¬ 
sarily  reflect  “real”  spatial  abilities  in  the  sense 
of  holistic  processing  and  visualization  and  are 
also  affected  by  other  factors  like  reasoning 
abilities  and  knowledge.  These  considerations 
imply  that  these  tests  probably  do  not  differenti¬ 
ate  between  subjects  with  high  and  low  spatial 
abilities  as  intended  in  certain  cases.  This 
should  be  kept  in  view  especially  when  spatial 
ability  tests  are  used  for  purposes  like  selection 
or  career  choice  procedures. 

In  the  second  study  with  79  students  (44  fe¬ 
males,  33  males,  2  unknown;  aged  16-19,  1l“^ 
and  12*’  grade)  we  added  the  JLO,  which  is 
usually  used  for  clinical  neurological  assess¬ 
ment,  to  the  test  battery.  This  test  examines  the 
ability  to  estimate  angular  relationships  between 
line  segments  and  serves  as  an  instrument  to 
measure  skills  for  recognition  and  processing  of 
simple  visual  stimuli.  We  can  only  present  pre¬ 
liminary  results  of  this  study  yet  but  it  points  to 
very  interesting  gender  specific  differences  in 
strategies  used  for  solving  spatial  tests. 

If  we  look  at  the  test  scores  separated  by  gen¬ 
der  we  see  that  in  case  of  male  students  almost 
all  spatial  ability  scores  are  significantly  corre¬ 
lated  but  performance  in  a  logical  reasoning  test 
(WIT  Letter  Sequence  subtest)  is  not  correlated 
to  any  of  them.  In  the  female  students,  almost 


Table  3.  Component  matrix  for  female  partici¬ 
pants 

all  spatial  ability  tests  are  significantly  correlated 
both  to  each  other  and  to  the  logical-reasoning 
test. 

Separate  factor  analyses  for  male  (Table  2)  and 
female  (Table  3)  students  identified  two  factors 
in  both  cases,  however,  factor  loadings  are  dif¬ 
ferent  for  men  and  women. 

For  men  (Table  2)  PSVTR,  MCT,  MRT,  DAT, 
and  JLO  load  on  factor  1,  which  is  clearly  a 
“spatial”  factor,  whereas  only  logical  reasoning 
loads  on  factor  2.  For  women  (Table  3), 
PSVTR,  MCT,  and  DAT,  which  are  complex 
spatial  tasks,  load  on  factor  1,  together  with 
logical  reasoning.  MRT  and  JLO,  which  focus 
on  elementary  spatial  functions,  load  on  factor 
2. 

VWien  logical  reasoning  is  partialled  out  of  the 
correlations  among  the  spatial  tests,  the  correla¬ 
tions  in  the  male  sample  do  not  change  much, 
whereas  in  the  females,  almost  all  correlations 
among  spatial  tests  disappear.  The  reverse  pat¬ 
tern  shows  when  JLO  performance  is  partialled 
out;  In  the  male  sample,  all  correlations  among 
spatial  tests  disappear,  while  in  the  female  sam¬ 
ple,  only  the  correlations  of  the  MRT  to  the 
other  spatial  tests  disappear  and  all  correlations 
to  logical  reasoning  get  stronger. 

These  preliminary  results  suggest  that  in  female 
students,  high  spatial  performance  is  related  to 
good  reasoning  skills  and  logical  thinking, 
whereas  for  male  students,  speed  and  accuracy 
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of  basic  spatial  processes  are  the  best  predictor 
of  spatiai  test  performance. 

FUTURE  WORK 

Within  the  next  few  months  our  evaluation  re¬ 
garding  spatial  intelligence,  as  described  in  pre¬ 
vious  sections,  will  be  conducted.  In  the  future 
we  plan  to  study  general  improvements  of  spa¬ 
tial  ability  in  contrast  to  task  specific  improve¬ 
ments  in  detail.  We  want  to  seek  for  an  answer 
to  the  question  if  direct  manipulation  as  used  in 
ConstructSD  and  explorative  interaction  is  suffi¬ 
cient  to  improve  spatial  understanding  and  last 
but  not  least  if  training  of  spatial  intelligence  in  a 
virtual  environment  is  more  “efficient”  than  in  a 
real  environment. 

Further,  we  plan  to  utilize  ConstructSD  as  a  tool 
for  evaluating  various  aspects  in  virtual  environ¬ 
ments  in  our  future  research.  For  example  a 
comprehensive  pedagogic  evaluation,  evaluat¬ 
ing  e.g.  teaching  styles  or  transfer  of  learning  in 
educational  VR/AR  applications  would  be  inter¬ 
esting. 

ACKNOWLEDGEMENTS 

The  authors  thank  Istvan  Barakonyi  for  his  help 
during  the  evaluation  studies.  We  thank  ail  par¬ 
ticipants  of  evaluations  as  well  as  other  teach¬ 
ers  and  students  for  testing  and  giving  useful 
feedback.  Part  of  this  research  was  funded  by 
the  Austrian  Science  Fund  (FWF)  contract  no. 
P14470,  P16803  and  Y193  and  by  the  EU  1ST 
project  Lab@Future  (I ST-200 1-34204). 

REFERENCES 

[1]  J.  T.  Bell  and  H.  S.  Fogler,  "The  Investiga¬ 
tion  and  Application  of  Virtual  Reality  as  an 
Educational  Tool,"  American  Society  for 
Engineering  Education  1995  Annual  Con¬ 
ference,  1995. 

[2]  G.  K.  Benett,  H.  G.  Seashore,  and  A.  G. 
Wesman,  "Differential  Aptitude  Tests, 
Forms  S  and  T."  New  York:  The  Psycho¬ 
logical  Corporation,  1973. 

[3]  A.  L.  Benton,  K.  Hamsher,  N.  R.  Varney, 
and  O.  Spreen,  Contributions  to  neurologi¬ 
cal  assessment:  A  clinical  manual.  New 
York:  Oxford,  1983. 

[4]  L.  N.  Brown,  C.  J.  Lahar,  and  J.  L.  Mosley, 
"Age  and  gender-related  differences  in 


strategy  use  for  route  information:  A  "map- 
present"  direction-giving  paradigm,"  Envi¬ 
ronment  and  Behavior,  vol.  30,  pp.  123- 
143,  1998. 

[5]  C.  Byrne,  "Water  on  Tap:  The  Use  of  Virtual 
Reality  as  an  Educational  Tool,"  University 
of  Washington,  College  of  Engineering, 
Washington  1996. 

[6]  J.  B.  Carroll,  Human  cognitive  abilities.  A 
survey  of  factor-analytic  studies.  Cam¬ 
bridge,  UK:  Cambridge  University  Press, 
1993. 

[7]  CEEB  College  Entrance  Examination 
Board,  "Special  Aptitude  Test  in  Spatial 
Relations  MCT,"  CEEB  College  Entrance 
Examination  Board,  1939. 

[8]  R.  P.  Darken,  T.  Allard,  and  L.  Achilla, 
"Spatial  Orientation  and  Wayfinding  in 
Large-Scale  Virtual  Spaces:  An  Introduc¬ 
tion,"  Presence:  Teleoperators  &  Virtual 
Environments,  vol.  7,  pp.  101-107,  1998. 

[9]  N.  Durlach,  G.  Allen,  R.  Darken,  R.  L.  Gar¬ 
nett,  J.  Loomis,  J.  Templeman,  and  T.  E. 
von  Wiegand,  "Virtual  environments  and  the 
enhancement  of  spatial  behavior:  Towards 
a  comprehensive  research  agenda,"  PRES¬ 
ENCE  -  Teleoperators  and  Virtual  Environ¬ 
ments,  vol.  9,  pp.  593-615,  2000. 

[10]  G.  Gittler  and  J.  Gluck,  "Differential  Trans¬ 
fer  of  Learning:  Effects  of  Instruction  in  De¬ 
scriptive  Geometry  on  Spatial  Test  Perform¬ 
ance,"  Journal  of  Geometry  and  Graphics, 
vol.  2,  pp.  71-84,  1998. 

[1 1]  J.  Gluck  and  S.  Fitting,  "Spatial  strategy 
selection:  Interesting  incremental  informa¬ 
tion,"  International  Journal  of  Testing,  vol.  in 
press,  2003. 

[12]  R.  B.  Guay,  "Purdue  Spatial  Visualization 
Test:  Rotations,"  Purdue  Research  Founda¬ 
tion,  West  Lafayette,  IN  1977. 

[13]  R.  E.  Guttman,  E.  Epstein,  M.  Amir,  and  L. 
Guttman,  "A  structural  theory  of  spatial 
abilities,"  Applied  Psychological  Measure¬ 
ment,  vol.  14,  pp.  217-236,  1990. 

[14]  O.  A.  Jager  and  K.  Althoff,  Der  WILDE- 
Intelligenz-Test.  (WIT);  e. 
Strukturdiagostikum;  Handanweisung. 
Gottingen:  Hogrefe,  Verlag  fiir  Psychologie, 
1994. 

[15]  O.  A.  Jager,  H.-M.  SuB,  and  A.  Beauducell, 
"Berliner  Intelligenzstruktur-Test." 
Gottingen:  Hogrefe,  Verlag  fur  Psychologie, 
1997. 


74 


Kaufmann  et  al. 


[16]  H.  Kaufmann,  "Geometry  Education  with 
Augmented  Reality.,"  in  Ph.D.  Thesis.  Vi¬ 
enna:  Vienna  University  of  Technology, 
2004,  pp.  179. 

[17]  H.  Kaufmann  and  D.  Schmalstieg, 
"Mathematics  and  geometry  education  with 
collaborative  augmented  reality,"  Com¬ 
puters  &  Graphics,  vol.  27,  pp.  339-345, 
2003. 

[18]  H.  Kaufmann,  D.  Schmalstieg,  and  M.  Wag¬ 
ner,  "Construct3D:  a  virtual  reality  applica¬ 
tion  for  mathematics  and  geometry  educa¬ 
tion,"  Education  and  Information  Technolo¬ 
gies,  vol.  5,  pp.  263-276,  2000. 

[19]  M.  Kozhevnikov  and  M.  Hegarty,  "A  disso¬ 
ciation  between  object  manipulation,  spatial 
ability  and  spatial  orientation  ability,"  Mem¬ 
ory  &  Cognition,  vol.  29,  pp.  745-756,  2001. 

[20]  J.  J.  LaViola  Jr.,  "A  discussion  of  cybersick¬ 
ness  in  virtual  environments,"  ACM  SIGCHI 
Bulletin,  vol.  32,  pp.  47-56,  2000. 

[21]  C.  A.  Lawton,  "Gender  differences  in  way¬ 
finding  strategies;  Relationship  to  spatial 
ability  and  spatial  strategies,"  Sex  Roles, 
vol.  30,  pp.  765-779,  1994. 

[22]  B.  Loftin,  M.  Engelberg,  and  R.  Benedetti, 
"Applying  virtual  reality  in  education:  A  pro¬ 
totypical  virtual  physics  laboratory,"  in  Pro¬ 
ceedings  of  the  IEEE  1993  Symposium  on 
Research  Frontiers  in  Virtual  Reality.  Los 
Alamitos,  CA:  IEEE  Computer  Society 
Press,  1993,  pp.  67-74. 

[23]  D.  F.  Lehman,  "Spatial  ability:  A  review  and 
reanalysis  of  the  correlational  literature," 
Stanford  University  School  of  Education, 
Aptitude  Research  Project,  Stanford,  CA 
Tech.  Rep.  No.  8,  1979. 

[24]  D.  F.  Lehman  and  P.  C.  Kyllonen, 

"Individual  differences  in  solution  strategy 
on  spatial  tasks,"  in  Individuai  differences  in 
cognition,  D.  F.  Dillon  and  R.  R.  Schmeck, 
Eds.  New  York:  Academic  Press,  1983,  pp. 
105-135. 

[25]  F.  Mantovani,  'VR  Learning:  Potential  and 
Challenges  for  the  Use  of  3D  Environments 
in  Education  and  Training,"  in  Towards  Cy- 
berPsychology:  Mind,  Cognitions  and  Soci¬ 
ety  in  the  Internet  Age,  G.  Riva  and  C. 
Galimberti,  Eds.  Amsterdam:  lOS  Press, 
2001. 

[26]  J.  Prumper,  "Der  Benutzungsfragebogen 
ISONORM  9241/10;  Ergebnisse  Zur 
Reliabilitat  und  Validitat,"  in  Software- 


Ergonomie  '97,  R.  Liskowsky,  Ed.  Stuttgart, 

1997. 

[27]  C.  Quaiser-Pohl  and  W.  Lehmann,  "Der 
Mental-Rotation-Test,"  Swets  Test  Ser¬ 
vices,  Frankfurt/Main  2002. 

[28]  A.  A.  Rizzo,  J.  G.  Buckwalter,  J.  S.  MCGee, 
T.  Bowerly,  C.  van  der  Zaag,  U.  Neumann, 
M.  Thiebaux,  L.  Kim,  L.  Pair,  and  C.  Chua, 
"Virtual  environments  for  assessing  and 
rehabilitating  cognitive/functional  perform¬ 
ance,"  Presence  -  Teleoperators  and  Virtual 
Environments,  vol.  10,  pp.  359-374,  2001. 

[29]  A.  A.  Rizzo,  J.  G.  Buckwalter,  U.  Neumann, 
C.  Kesselman,  M.  Thiebaux,  P.  Larson,  and 
A.  Van  Rooyen,  "The  Virtual  Reality  Mental 
Rotation  Spatial  Skills  Project,"  CyberPsy- 
chology  and  Behavior,  vol.  1,  pp.  113-120, 

1998. 

[30]  D.  Schmalstieg,  A.  Fuhrmann,  G.  Hesina, 

Z.  S.  Szalavari,  L.  M.  Encarnacao,  M.  Ger- 
vautz,  and  W.  Purgathofer,  "The  Studier- 
stube  augmented  reality  project,"  Presence 
-  Teleoperators  and  Virtual  Environments, 
vol.  11,  pp.  33-54,  2002. 

[31]  K.-H.  Schmidt  and  P.  Metzler, 
Wortschatztest.  Weinheim:  Belz,  1992. 

[32]  E.  Souvignier,  "Training  raumlicher 
FShigkeiten.  [Training  spatial  abilities.],"  in 
Handbuch  Kognitives  Training,  K.  J.  Klauer, 
Ed.  Gottingen;  Hogrefe,  2001,  pp.  293-319. 

[33]  H.  Stumpf  and  J.  Eliot,  "A  structural  analy¬ 
sis  of  visual  spatial  ability  in  academically 
talented  students,"  Learning  and  Individual 
Differences,  vol.  11,  pp.  137-151,  1999. 

[34]  G.  Taxen  and  A.  Naeve,  "CyberMath:  Ex¬ 
ploring  Open  Issues  in  VR-Based  Learn¬ 
ing,"  SIGGRAPH  2001  Educators  Program, 
vol.  SIGGRAPH  2001  Conference  Ab¬ 
stracts  and  Applications,  pp.  49-51,  2001. 

[35]  F.  Tendick,  M.  Downes,  T.  Goktekin,  M.  C. 
Cavusoglu,  D.  Feygin,  X.  Wu,  R.  Eyal,  M. 
Hegarty,  and  L.  W.  Way,  "A  Virtual  Environ¬ 
ment  Testbed  for  Training  Laparoscopic 
Surgical  Skills,"  Presence,  vol.  9,  pp.  236- 
255,  2000. 

[36]  S.  G.  Vandenberg  and  A.  R.  Kuse,  "Mental 
Rotations:  a  group  test  of  three-dimensional 
spatial  visualization,"  Perceptuat  and  Motor 
Skills,  vol.  47,  pp.  599-604,  1978. 


Annual  Review  of  CyberTherapy  and  Telemedicine 


Contact 

Hannes  Kaufmann 

Institute  of  Software  Technology  and  Interactive 
Systems 

Vienna  University  of  Technology 
Favoritenstr.  9-11/188-2, 

A-1040  Vienna,  Austria 

Email:  kaufmann@ims.tuwien.ac,at 


76 


Annual  Review  of  CyberTherapy  and  Telemedicine 


The  Gedanken  Experiment  of  Human  Dynamics 

Y.  Cai^  and  J.  Klein-Seetharaman^'^ 

^CyLab,  Carnegie  Mellon  University,  Pittsburgh 
^  Department  of  Pharmacology,  University  of  Pittsburgh  Medical  School,  Pittsburgh 


Abstract;  Understanding  human  dynamics  is  an  important  step  in  ciinicai  therapies,  such  as  blood 
glucose  management  or  weight  control.  It  requires  insight  into  the  complex  interactions  between  the 
components  of  biomedical  systems.  This  is  challenging  because  of  the  enormous  amount  of  data 
and  knowledge  in  this  domain.  Gedanken  Experiment  is  an  intuitive,  qualitative  thinking  process, 
which  have  been  used  by  physicists  for  decades.  Our  goal  is  to  enable  an  interactive  game-like  envi¬ 
ronment  for  users  with  any  background,  while  remaining  sufficiently  flexible  to  target  medical  prob¬ 
lems  at  a  level  of  abstraction,  from  the  conformational  changes  of  a  protein  to  the  interaction  of  the 
various  biochemical  pathways  in  our  body.  Here,  we  present  an  interactive  and  visual  problem¬ 
solving  environment  for  the  biomedical  domain.  We  designed  a  biological  world  model,  in  which  us¬ 
ers  can  explore  biological  interactions  by  role-playing  “characters"  such  as  cells  and  molecules  or  as 
an  observer  in  a  "shielded  vessel",  both  with  the  option  of  networked  collaboration  between  simulta¬ 
neous  users.  The  system  architecture  of  these  "characters"  contains  four  main  components:^  Bio¬ 
behavior  is  modeled  using  cellular  automata.^  Bio-rhorphinq  uses  vision-based  shape  tracking  tech¬ 
niques  to  learn  from  recordings  of  real  biological  dynamics?  Bio-sensing  is  based  on  molecular  prin¬ 
ciples  of  recognition  to  identify  objects,  environmental  conditions  and  progression  in  a  process.''  Bio¬ 
dynamics  implements  mathematical  models  of  cell  growth  and  fluid-dynamic  properties  of  biological 
solutions.  The  principles  are  implemented  in  a  simple  world  model  of  the  human  vascular  system  and 
a  biomedical  problem  that  involves  an  infection  by  Neisseria  meningitides  where  the  biological  char¬ 
acters  are  white  and  red  blood  cells  and  Neisseria  cells.  Our  case  studies  show  that  the  system  can 
be  used  for  public  health  education,  biomedical  simulation  and  creative  problem  solving. 


INTRODUCTION 

Understanding  human  dynamics  is  an  important 
step  in  clinical  therapies,  such  as  blood  glucose 
management  or  weight  control.  It  requires  in¬ 
sight  into  the  complex  interactions  between  the 
components  of  biomedical  systems.  This  is 
challenging  because  of  the  enormous  amount  of 
data  and  knowledge  in  this  domain.  Gedanken 
Experiment  (GE)  is  an  intuitive,  qualitative  think¬ 
ing  process,  which  have  been  used  by  physi¬ 
cists  for  decades.  Novel  approaches  are  being 
developed,  for  example  a  storytelling  system 
has  been  presented  to  fertilize  multidisciplinary 
biomedical  problem  solving.®  Furthermore,  mod¬ 
ern  biomedical  education  makes  extensive  use 
of  visualization  of  biomedical  processes  and 
concepts,  for  example  the  publication  of  the  hu¬ 
man  genome  sequence  was  accompanied  by  a 
CD-ROM  that  presented  genome  background 
as  well  as  DNA  sequencing  techniques  in  an¬ 
imations.®  However,  these  visualization  tools 
are  mostly  designed  to  complement  traditional 
teaching  techniques  and  are  not  very  interac¬ 


tive.  More  interaction  is  provided  in  the  “Virtual 
Cell”,  a  virtual  environment  in  which  question- 
based  assignments  are  given  to  users  in  a 
simulated  laboratory.^  A  submarine  is  launched 
that  immerses  the  user  in  the  virtual  environ¬ 
ment  of  the  cell  populated  by  sub-cellular  com¬ 
ponents  which  the  user  can  investigate.  With  a 
toolbox,  various  cellular  processes  can  be  in¬ 
vestigated  experimentally.  The  results  of  these 
investigations  and  experimentations  allow  users 
to  solve  the  assignments  at  their  own  pace  and 
through  their  own  motivation.  It  was  shown  that 
this  approach  significantly  improves  authentic 
learning,  in  particular  for  large  enrollment  gen¬ 
eral  biology  classes.®  At  the  other  end  of  the 
spectrum,  realistic,  fully  interactive  virtual  labo¬ 
ratories  have  been  developed  to  simulate 
chemical,®  biomedical,^®  and  recently 
nanoscience^^  laboratory  experiments.  How¬ 
ever,  in  their  present  form,  these  systems  were 
not  designed  to  be  integrated  GEs  because 
they  are  targeted  to  students  and  researchers 
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for  solving  very  specific  problems  and  therefore 
contain  a  large  amount  of  domain  specific  infor¬ 
mation.  Inexperienced  users  will  not  have  suffi¬ 
cient  insight  needed  for  discovery  and  solving 
problems.  Insight,  i.e.  the  capability  to  make 
non-obvious  connections  between  the  complex 
interactions  of  the  components  of  these  sys¬ 
tems,  is  the  main  requirement  for  solving  bio¬ 
medical  problems.^^  Such  Insightful  solutions 
can  often  be  found  in  an  interactive  and  visual 
PSE,  as  demonstrated  for  example  by  the  fact 
that  despite  the  modern  numerical  computing 
technologies,  biophysicists  today  still  use  Ge- 
danken  experiments  for  concept  development.^^ 
Although  there  are  many  virtual  reality  three- 
dimensional  molecular  models  available,  bio¬ 
chemists  still  use  hand-made  models  for  intui¬ 
tive  reasoning.  It  is  striking  that  simple  intuitive 
simulation  is  still  one  of  the  most  powerful  ap¬ 
proaches  to  creative  problem  solving. 

Since  the  early  days  of  artificial  intelligence, 
issues  of  modeling  scientific  reasoning  and  its 
representation,  in  particular  for  those  connected 
with  everyday  knowledge  of  the  behavior  of  the 
physical  world,  have  been  studied. At  least 
two  aspects  have  been  explored:  multiple  repre¬ 
sentation  and  qualitative  reasoning.  Computa¬ 
tion  with  Multiple  Representations  (CaMeRa)  is 
a  model  that  simulates  human  problem  solving 
with  multiple  representations,  including  pictures 
and  words.^®  CaMeRa  combines  a  parallel  net¬ 
work,  used  to  process  the  low-level  pictorial  in¬ 
formation,  with  rule-based  processes  in  higher- 
level  pictorial  and  verbal  reasoning.  Further¬ 
more,  many  Al  systems  have  been  developed 
to  simulate  the  cognition  about  physical  and 
biological  knowledge.  What  will  happen  if  we 
spill  a  glass  of  milk  on  the  floor?  For  humans, 
the  answer  is  common  sense,  but  understand¬ 
ing  this  process  is  non-trivial  for  computers.  To 
arrive  at  an  exact  solution,  the  computer  has  to 
solve  a  set  of  non-linear  partial  differential  equa¬ 
tions  of  hydrodynamics  that  are  computationally 
intractable  even  for  simple  boundary  condi¬ 
tions.^®  A  few  studies  have  focused  on  the  quali¬ 
tative  simulation  of  physical  phenomena.  Thus, 
Gardin  uses  two-dimensional  diagrams  to  repre¬ 
sent  physical  objects  and  their  interaction^^  and 
Forbus  uses  the  fuzzy  language  of  "qualitative 
physics"  to  model  the  physical  variables.^® 
Lower-resolution  qualitative  models  have  made 
significant  impact  in  many  fields,  including  biol¬ 


ogy.  A  typical  example  is  the  Game  of  Life,  a 
"Cellular  Automaton."^®  A  cellular  automaton  is 
an  array  of  identically  programmed  automata,  or 
"cells",  which  interact  with  one  another.  The 
state  of  each  cell  changes  from  one  generation 
to  the  next  depending  on  the  state  of  its  immedi¬ 
ate  neighbors.  By  building  appropriate  rules, 
complex  behavior  can  be  simulated,  ranging 
from  the  motion  of  fluids  to  outbreaks  of  starfish 
on  a  coral  reef.  Even  if  the  line  of  cells  starts 
with  a  random  arrangement  of  states,  the  rules 
force  patterns  to  emerge  In  life-like  behavior. 
Empirical  studies  by  Steven  Wolfram^®  and  oth¬ 
ers  show  that  even  the  simple  linear  automata 
behave  in  ways  reminiscent  of  complex  biologi¬ 
cal  systems.  In  light  of  this  discovery,  we  intend 
to  use  simple  biological  characters  to  generate 
dynamic  interactions. 

Creative  Problem  Solving  Environments  have 
been  studied  mainly  in  the  management  sci¬ 
ence  area.  Most  recent  studies  focus  on  col¬ 
laborative  creativity,  stimulus  culture,  and  infor¬ 
mation  flow.  For  example,  “brainstorming”  has 
been  viewed  as  a  panacea  in  corporations  since 
IDEO  has  promoted  the  methodology.®’  John 
Kao’s  “Jamming”  theory®®  uses  a  jazz  jam  ses¬ 
sion  as  a  metaphor  to  address  how  to  stimulate 
employees  at  work.  Steven  Eppinger’s  Design 
Information  Matrix®®  focuses  on  representing 
information  flows  rather  than  task  flows.  Accord¬ 
ing  to  Constructionism,  people  do  not  simply 
“get  an  idea”;  they  construct  it.  Within  the  frame¬ 
work  of  the  recently  developed  “Idea  Flow”  the¬ 
ory  to  address  the  dynamics  of  creative  problem 
solving, ®‘’  it  was  shown  that  the  bidirectional 
idea  flow  is  the  most  efficient  interaction  pattern 
in  innovation.  Feedback  can  enhance  the  flow 
both  in  terms  of  effectiveness  and  efficiency. 
Multiphysics  simulation  and  knowledge-based 
innovation  heuristics  build  the  bridge  between 
the  innovative  idea  and  reality,  which  will  even¬ 
tually  change  the  landscape  of  scientific  discov¬ 
ery.  Open  source,  open  system  architecture  and 
virtual  communities  bring  idea  flows  from  the 
outside.  This  is  the  trend  of  contemporary  scien¬ 
tific  discovery  and  system  design  that  not  only 
creates  just  a  product  but  an  inspiration  for  in¬ 
teraction. 

Here  we  present  a  computer  game  as  a  novel 
environment  for  biological  problem  solving, 
where  it  provides  a  real-time  interactive  platform 
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Figure  1.  World  model  and  biological  characters  implemented  in  the  game.  A.  Wireframe  model  of 
the  vascular  system  world  model.  B.  3-D  photorealistic  model  showing  arteries  and  capillaries.  C. 
The  Macrophage  (white  blood  cell)  is  inside  the  blood  stream  with  red  blood  cells.  D.  After  actively 
moving  out  of  the  blood  stream,  the  macrophage  approaches  bacteria  that  infected  human  body  tis¬ 
sue. 


for  users.  The  goal  of  this  study  is  to  develop  a 
game-based  PSE  for  users  to  explore  multi¬ 
modal  interactions  inside  a  biological  system.  It 
includes  essential  biological  simulation  models 
for  the  immune  system  and  the  blood  system.  It 
allows  users  to  manipulate  and  to  participate  in 
the  interactions  among  the  components  of  the 
system.  The  biological  characters  are  simulated 
by  software  agents. 

PROTOTYPE 

As  a  test  bed  for  the  development  of  a  game- 
based  Gedanken  experiment  system  for  bio¬ 
medical  science,  we  designed  a  scientific  prob¬ 
lem  that  is  derived  from  our  ongoing  research 
projects.  By  applying  computational  language 
technologies  to  the  large  amounts  of  whole  ge¬ 
nome  sequence  data  publicly  available,  we  have 
identified  “genome  signatures”  that  may  provide 
novel  approaches  to  the  development  of  vac¬ 
cines  against  pathogenic  microorganisms  such 
as  Neisseria.^  The  biomedical  problem  to  be 
explored  here  is  to  find  treatment  for  fatal  menin¬ 
gitis.  The  new  idea  for  the  biomedical  simulator  is 


to  develop  an  interactive  interface  modeled  after 
traditional  game  engines  that  teaches  users  with¬ 
out  background  in  biology  the  understanding  of 
this  research  problem  ranging  in  hierarchy  from 
atomic  to  macroscopic  scales  (Fig.  1).  In  this  hi¬ 
erarchy,  the  macroscopic  level  is  that  of  infection 
of  a  human  body  with  Neisseria.  Fighting  the  in¬ 
fection  requires  a  molecular  level  understanding 
of  the  processes  involved.  The  goal  is  to  provide 
the  user  with  the  necessary  insight  to  creatively 
generate  and  test  possible  approaches  to  solving 
this  problem  using  the  simulator. 

BioSim  version  1 .0  is  a  rapid  prototype  for  this 
system.  It  is  a  two-stage  game  that  simulates 
the  Journey  of  red  blood  cells  and  white  blood 
cells  (macrophages).  The  goal  is  to  introduce 
the  basic  concepts  of  cellular  interaction  and  the 
human  immune  system.  The  game  begins  with 
an  animated  scene  of  a  blood  stream  with  red 
and  white  cells  moving  passively  with  the  heart¬ 
beat.  Using  a  mouse  and  the  arrow  keys,  the 
player  can  take  the  role  of  a  biological  charac¬ 
ter,  for  example  a  macrophage,  and  navigate 
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inside  the  blood  stream.  The  user  can  actively 
move  to  tissue  that  is  infected  by  bacterial  cells, 
which  multiply  at  a  certain  speed.  Screen  shots 
of  these  processes  are  shown  in  Fig.  2. 

BioSim  1.0  is  implemented  on  PC.  Photorealis¬ 
tic  3D  models  of  components  of  the  system 
were  created  with  3D  Studio  Max  and  exported 
to  Game  Studio  3D  (an  example  is  shown  in 
Fig.  3).  The  3D  Modeler  of  GameStudio  3D  was 
used  to  create  the  game  scenes,  biomorphed 
characters  and  the  integration  of  the  world/ 
character  dynamics  and  interactions.  C-script,  a 
C-styie  language,  was  used  to  encode  the  bio¬ 
dynamics  and  bio-sensing  behaviors.  Game 
Studio  is  run  under  the  Windows  operation  sys¬ 
tem.  It  provides  capability  for  either  single  user 
or  multiple  users  across  the  Internet. 

BIOLOGICAL  “WORLD  MODEL” 

In  game  design,  “world  models”  are  similar  to 
the  theatre  stage  or  film  scene  with  which  actors 
and  characters  interact.  A  world  model  is  often 
static  and  large  in  size.  In  this  project,  we  devel¬ 
oped  a  comprehensive  world  model  that  in¬ 
cludes  the  vascular  system  with  artery,  veins 
and  capillaries,  as  well  as  tissues  (Fig.  1  A,B). 
In  this  world,  the  user  can  fly,  walk  or  run 
through  as  one  of  the  biological  characters  (Fig. 

1  C,D).  In  the  prototype  BioSim  1.0,  we  have 
developed  two  scenes:  inside  and  outside  of  the 
capillary.  The  transition  of  the  scenes  is  possi¬ 
ble  by  “squeezing”  a  character  actively  from  the 
blood  stream  to  the  tissue  in  the  capillary  re¬ 
gions. 

BIOLOGICAL  CHARACTERS 

We  have  defined  the  following  3D  animated 
characters  that  simulate  biological  behavior: 
bacteria,  macrophages,  and  red  blood  cells.  To 
define  interactions  among  characters  and  be¬ 
tween  the  characters  and  the  world  model,  we 
use  collision  detection  (section  2.4.3  below).  For 
the  stand-alone  characters,  we  apply  bio¬ 
morphing  to  assign  key  frames  to  them  (section 
2.4.2  below).  Bio-morphing  is  accomplished  by 
digitizing  deformed  shapes  from  microscopic 
images  of  organisms,  building  wire  frames  and 
attaching  texture  and  color  skins. 


SYSTEM  ARCHITECTURE 

The  PSE  contains  three  interaction  modes:  role- 
play,  voyage  and  networked  problem  solving. 

Role-Play.  The  system  allows  the  user  to  be  a 
biological  character  in  the  game.  Cognition  Sci¬ 
ence  shows  that  role-play  is  an  important  way  to 
stimulate  creative  ideas.  It  enables  the  user  to 
have  an  intimate  connection  to  the  character. 
Also,  personalization  of  a  biological  character 
makes  a  game  more  interactive. 

Voyage.  The  user  can  navigate  through  the  bio¬ 
logical  system  in  the  game,  either  as  a  character, 
or  using  a  ‘ship’,  supporting  different  view  angles, 
e.g.  traveling  through  capillaries  and  tissues. 
Voyage  allows  exploration  at  the  user’s  chosen 
leisure,  accommodating  users  with  various  back¬ 
grounds. 

Distributed  Problem  Solving.  The  game  en¬ 
gine  allows  users  to  play  the  game  over  the  Inter¬ 
net  so  that  large  problems  can  be  solved  collabo- 
ratively  or  antagonistically,  e.g.  some  users  can 
play  macrophages  and  others  can  play  bacteria. 
The  distributed  problem  solving  enables  diverse 
game  strategies  and  more  excitement  of  the 
game.  The  user  can  also  choose  between  two 
aims,  rather  than  playing  the  role  of  a  single  bio¬ 
logical  component  only.  The  user  can  assume 
the  roles  of  multiple  biological  characters,  thus 
studying  their  individual  influence  on  a  particular 
aim.  These  aims  are  to  induce  an  infection  with 
Neisseria  and  ensure  its  successful  propagation 
in  the  human  body  or  to  fight  the  Neisseria  infec¬ 
tion. 

BIOLOGICAL  CHARACTER  LIBRARY 

A  biological  character  is  defined  as  an  entity 
that  includes  functions,  forms,  behaviors  and 
interfaces.  For  exampie,  some  of  the  functions 
of  a  macrophage  are  to  iocate  and  destroy  bac¬ 
teria  (see  below),  while  the  function  of  a  bacte¬ 
rium  is  to  divide  and  spread.  A  given  character 
has  a  sequence  of  forms  in  correspondence  to 
its  dynamic  behaviors.  An  interface  to  interact 
with  other  entities  or  the  environment  is  de¬ 
signed. 

Example  Macrophage: 

Functions  =  locate  bacteria  |  destroy  bacteria 
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Behaviors  =  walk  \  eat  |  shrink  \  kill  \  die 
Forms  (key-frame  sequences)  =  moving  \ 
search  \  squeeze  \  killing  \  death 
Interfaces  =  collision  \  chemotaxis  (sensing)  \ 
tactile  (sensing) 

The  principles  describing  behaviors,  forms  and 
interfaces  that  are  common  to  all  characters  are 
described  below. 

Behavior.  Interaction  is  the  key  to  computer 
games,  and  we  believe  also  to  an  efficient  bio¬ 
medical  PSE.  We  therefore  allow  the  user  to 
control  the  behavior  of  biological  characters  by 


realistic  and  scientifically  accurate  bio¬ 
interactions.  The  transitions  of  each  character 
are  represented  by  a  state  machine  (Fig.  4A). 
For  example  a  macrophage’s  states  include  the 
transitions  to  deform,  shrink,  eat,  walk  and  die. 
Each  behavior  is  defined  by  a  set  of  forms  (see 
section  2.4.2  below).  We  simulate  biological 
processes  and  character  behavior  (“bio¬ 
dynamics”)  realistically.  For  each  character,  we 
define  its  interactive  modes,  such  as  motion, 
reproduction  and  death.  Taking  bacteria  for  ex¬ 
ample,  we  use  the  following  rules  (Fig.  4B): 


A.  B.  C. 


Figure  2  Examples  for  Character  Dynamics.  A.  State  machine  for  a  macrophage.  B.  Dynamics  of  an 
organism  modeled  by  Cellular  Automata.  C.  Example  for  the  time-evolution  of  bacterial  growth  and 
spatial  distribution. 


Time  x  1 0  [seconds]  Time  x  1 0  [seconds] 

Figure  3.  Strategies  against  bacterial  Infection  that  can  be  explored  in  the  game,  speed  of  macro¬ 
phage  movement  (left)  and  use  of  antibiotics  (right).  The  fast  pace  strategy  often  leads  to  missing 
targets.  The  slow  pace  strategy  gains  steady  capture  rate.  The  use  of  antibiotics  can  be  limited  by 
gradual  development  of  bacterial  resistance.  At  that  point,  administration  of  drug  does  not  inhibit 
bacterial  growth. 
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Autonomous  motion.  Given  a  fixed  duration, 
each  bacterium  moves  a  distance  x  at  angle  y, 
where  x  and  y  are  random  values.  The  distance 
should  not  exceed  the  maximal  distance. 

Reproduction.  Given  a  predefined  duration, 
each  bacterium  reproduces  a  copy  of  its  own 
which  is  placed  beside  its  original  position. 

The  Logistic  growth  modeF‘  adequately  de¬ 
scribes  the  reproduction  process  of  simple  or¬ 
ganisms  over  limited  time  periods  by  equation 
(1),  where  M  is  the  carrying  capacity  of  the 
population  B  at  time  n  with  growth  rate  r. 

6n+1  ~  fin  FBn("l  "  Br/M)  . 

(1) 

Death.  If  a  bacterium’s  life  cycle  is  over  or  if 
other  cells  eat  it,  it  is  removed  from  the  scene. 

PROBLEM  SOLVING  WITH  BIOSIM 

SUPPORTING  STRATEGIC  THINKING  IN  US¬ 
ERS 

We  conducted  experiments  in  the  effectiveness 
of  the  game  to  raise  an  awareness  of  the  impor¬ 
tant  issues  in  biomedical  research  on  users  with 
no  background.  The  ideal  group  at  this  stage  of 
implementation  of  the  game  is  young  children, 
for  two  reasons.  One,  children  are  unbiased  and 
without  background.  Second,  children  learn  op¬ 
timally  when  the  material  to  be  learned  is  pre¬ 
sented  to  them  in  an  accurate  way  to  avoid  the 
build-up  of  incorrect  models  by  implicit  learn- 
ing.^^  Implicit  learning  of  correct  biomedical  con¬ 
cepts  by  children  therefore  requires  the  same 
fundamental  issue  of  scientific  accuracy  as 
other  users  will  require  once  the  game  reaches 
the  stage  of  providing  a  PSE  for  users  with  any 


background.  We  tested  BioSim  1.0  on  14  chil¬ 
dren  at  KinderCare,  Cranberry,  PA,  on  August 
9th,  2002  and  February  25,  2003.  We  let  four- 
and  five-year-old  children  play  with  the  game  on 
a  laptop  and  focused  our  attention  on  strategic 
aspects  and  active  questioning. 

Two  strategies  were  quantified,  the  speed  of 
macrophage  movement  towards  the  bacteria 
(Fig.  3,  left)  and  the  use  of  antibiotics  in  aiding 
the  killing  of  the  bacteria  (Fig.  3,  right).  All  chil¬ 
dren  learnt  quickly  to  shift  from  fast  pace  chas¬ 
ing  to  slow  pace  chasing  so  that  their  capture 
rate  was  improved.  We  then  tested  a  more 
challenging  concept,  that  of  usage  of  antibiotics 
to  aid  the  killing  of  the  bacteria.  We  included  the 
ability  of  bacteria  to  develop  resistance  in  our 
growth  model.  Thus,  the  children  had  to  dis¬ 
cover  that  antibiotics  at  some  stage  in  the  game 
no  longer  inhibit  bacterial  growth.  This  was  only 
observed  by  a  single  5-year  old,  all  other  chil¬ 
dren  kept  on  administering  antibiotics  despite 
energy  consumption  and  lack  of  effect  (Fig.  8, 
right).  These  types  of  quantitative  assessment 
of  strategic  behavior  of  users  open  novel  ways 
to  analyze  learning  of  problem  solving  skills  that 
would  not  be  possible  with  conventional  teach¬ 
ing  methods. 

SUPPORTING  CREATIVE  THINKING  IN  US¬ 
ERS 

To  assess  learning  in  children,  we  asked  ques¬ 
tions  such  as  “How  does  the  macrophage  get 
out  of  the  capillary?”  or  “How  do  you  kill  bacte¬ 
ria?”  The  children  used  intuitive  metaphors,  for 
example  the  analogy  of  “vacuum”  and  “crash 
into”  to  describe  how  the  macrophage  attacks 
bacteria.  This  shows  that  the  players  are  very 
sensitive  to  the  intimate  design  details  of  the 
game,  which  opens  a  window  for  game  devel- 


Observation 

4-year  old 

5-year  old 

Asked  relevant  questions 

0 

4 

Controlled  the  game  successfully 

2 

5 

Described  bacterial  growth  behavior 

1 

5 

Described  macrophage  behavior 

2 

6 

Table  1,  Comparison  of  the  reactions  of  two  groups  of  children  to  the  game.  The  children  in  the  first 
group  were  4  years  old,  those  in  the  other  were  5  years  old.  Each  group  consisted  of  7  children,  and 
the  total  number  of  children  tested  was  14. 
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opens  to  encode  very  subtle  knowledge  about 
complex  biological  interactions.  Finally,  we 
tested  the  game-induced  stimulation  of  ques¬ 
tioning  and  creative  thinking  in  the  children.  The 
results  are  summarized  in  Table  1.  The  five- 
year-old  children  asked  several  meaningful 
questions,  for  example:  “Are  bacteria  germs?” 
“Where  do  the  white  cells  go?”  “What’s  a  red 
cell?”  “Where  do  the  bacteria  live?”  “Is  the 
macrophage  good  or  bad?”  Overall,  four-year- 
old  children  asked  fewer  questions,  and  most  of 
their  questions  were  not  relevant,  for  example, 
“Do  you  have  other  games?”  “I  don’t  want  my 
head  eaten  off.”  These  observations  suggest 
that  there  may  be  a  turning  point  between  ages 
4  and  5  where  a  PSE  can  become  effective. 

SUMMARY 

Future  biomedical  problem  solving  is  beyond 
traditional  means  because  of  the  existing  chal¬ 
lenges  in  cross-disciplinary  communication  and 
interpretation  and  utilization  of  vast  quantities  of 
available  biomedical  data.  We  want  to  build  a 
virtual  PSE  that  combines  advanced  computer 
graphics,  computer  vision,  artificial  intelligence 
technologies  and  creative  instruction  technolo¬ 
gies.  In  this  PSE,  cross-disciplinary  education 
will  be  on-demand,  entertaining  and  interactive. 
This  will  allow  focus  on  discovery  and  creativity 
rather  than  one-way  tutoring.  Towards  this  long¬ 
term  goal,  here,  we  have  presented  a  game- 
based  PSE,  where  users  can  explore  complex 
biological  interactions  with  navigation,  role-play, 
and  networked  collaboration.  The  study  investi¬ 
gates  the  system  architecture  of  the  biological 
game,  bio-morphing  characters,  and  bio¬ 
interactions  with  bio-sensing  and  bio-dynamics. 
The  game  is  based  on  realistic  biological  mod¬ 
els,  such  as  logistic  growth  models  of  simple 
organism  reproduction  and  immigration  models 
of  cell  movements.  The  prototype  has  been  im¬ 
plemented  on  PC  and  tested  in  a  preschool  en¬ 
vironment  where  users  have  little  knowledge  in 
biology.  The  experiment  shows  that  the  game 
greatly  inspired  users  both  in  concept  learning 
and  entertainment.  It  supports  strategic  thinking, 
creative  thinking,  as  well  as  professional  inno¬ 
vative  problem  solving.  Our  study  suggests  that 
the  game-based  PSE  helps  users  to  learn  bio¬ 
system  dynamics  and  multiple  object  interac¬ 
tions. 
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The  Recording  of  Observational  Behaviors  in  Virtual  Immersion:  A  New 
Research  and  Clinical  Tool  to  Address  the  Problem  of  Sexual  Preferences 

with  Paraphiliacs 
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^  Laboratoire  de  cyberpsychologie,  Departement  de  psychoeducation  et  de  psychologie,  Universite  du 
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^Departement  de  psychologie,  Universite  de  Montreal, 
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Abstract:  A  new  method  to  assess  and  treat  deviant  sexual  preferences  based  on  the  combined 
usage  of  virtual  reality,  eye-tracking  devices  and  penile  plethysmography  is  presented.  This  method 
tries  to  answer  some  of  the  shortcomings  undermining  current  methods  used  in  research  and  clinical 
practice  of  forensic  psychology  and  rehabilitation.  Among  these  shortcomings,  threats  to  internal 
validity  caused  by  non  compliance  and  faking  normal  sexual  responses  are  the  most  important 
ones.  The  victimization  of  actual  photographed  models  (adults  and  children)  to  prompt  sexual 
arousal  Is  also  ethically  problematic.  Hence  the  use  of  sexual  avatars  depicting  various  age  periods 
and  secondary  sexual  characteristics.  These  are  used  to  elicit  oculomotor  behaviors  and  physiologi¬ 
cal  responses  expressing  esthetic  interest,  approach  and  genital  responses  which  are  continuously 
monitored  via  video-oculography  and  penile  plethysmography.  Preliminary  results  of  the  avatars’ 
validation  process  are  also  presented. 


THEORETICAL  BACKGROUND 

It  is  on  sexual  preference  assessment’s  prem¬ 
ises  that  an  effective  treatment  of  sexual  devi- 
ancy  relies  first.  Sexual  preferences  are  usually 
assessed  either  by  resorting  on  sexual  re¬ 
sponses  recorded  with  a  penile  plethysmograph 
or  by  using  visual  reaction  time  from  stimuli  with 
sexual  content^ However,  numerous 
shortcomings  come  with  penile  plethysmogra¬ 
phy  and  visual  reaction  time,  and  most  notably 
not  conscientiously  paying  attention  to  stimuli 
and/or  exerting  a  misleading  voluntary  erectile 
control,  which  are  important  threats  to  the  inter¬ 
nal  validity  of  these  assessment  procedures  (do 
not  forget  that  these  patients  are  generally  not 
eager  for  being  assessed  for  such  reasons,  and 
in  such  ways^'^^'^'^.The  victimization  of  children 
is  another  major  weakness  with  assessment 
procedures  using  pictures  of  real  children,  ei¬ 
ther  to  arouse  deviant  sexual  response  or  devi¬ 
ant  interest  as  indexed  by  visual  reaction  time. 

In  order  to  get  round  these  limitations,  we  de¬ 
veloped  a  method  that  controls  gazing  activity 
relative  to  sexual  avatars.  We  had  already  used 
a  similar  logic  in  the  past  to  assess  behavioral 


avoidance  with  arachnophobics^°  and  to  assess 
sexual  preferences  with  normal  subjects^^.This 
method  counts  on  the  possibility  to  literally  get 
inside  a  patient’s  subjective  viewpoint  by  simul¬ 
taneously  considering  the  momentary  scene 
displayed  to  him  in  virtual  immersion  as  well  as 
his  punctual  oculomotor  activities,  as  these  lat¬ 
ter  largely  contribute  to  the  content  of  his  atten- 
tional  focus  (Figure  1;  ^^24,21,22^  Coupling  this 
new  method  with  classic  penile  plethysmogra¬ 
phy  allows  indeed,  prima  facie,  a  far  better  in¬ 
ternal  validity  to  sexual  preference  assessment, 
considering  the  aforementioned  first  two  short¬ 
comings  inherent  to  classic  methods. 

Regarding  the  stimulus  part  peculiar  to  this  new 
assessment  method,  avatars  depicting  naked 
characters  of  both  genders  and  of  clinically  sig¬ 
nificant  age  phases  are  required  to  prompt  sex¬ 
ual  attraction  and  arousal.  It  is  usually  theorized 
that  sexual  attraction  and  arousal  sequentially 
unfold  with  the  aesthetic,  the  approach  and  the 
genital  responses^'^®'^'*.  As  a  major  asset,  the 
resort  to  these  synthetic  characters  guards 
against  the  victimization  of  real  models  that  are 
used  with  classic  methods^®. 
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Figure  1.  Inside  the  patient’s  vantage  point  of  view;  the  aosshair  depicts  the  momentary  point  of  re¬ 
gard  of  the  immersed  subject  shown  in  the  upper  right  corner  photo;  the  avatar  personifies  a  female 
child  of  8  years  old;  on  the  right  hand  side,  a  photo  of  the  monocular  infrared  eye-tracking  system 
combined  within  the  binocular  HMD 


METHODS 

In  a  more  technical  fashion,  our  method  relies 
upon  a  technological  setting  including  what  is 
usually  necessary  to  present  virtual  environ¬ 
ments  in  immersion  plus  equipments  dedicated 
to  eye-movement  tracking  from  within  a  Virtual 
Research  V8  head  mounted  display  (HMD).  A 
special  mounting  built  from  an  ASL  monocular 
infrared  eye-tracking  system  combined  within  the 
binocular  HMD  is  used  to  track  eye-movements 
in  immersion  (see  figurel).  Head-movements 
are  recorded  from  an  Intersense  IS-900  tracking 
system  rendering  the  6  degrees-of-freedom 
(DOF)  of  translation  and  rotation. 

Our  method  performs  gaze  analysis  by  the  way 
of  virtual  measurement  points  (VMP)  placed  over 
virtual  objects  for  the  analysis  of  eye-movements 
in  relation  to  specific  features  of  these  objects. 
These  VMPs  are  placed  over  meaningful  areas 
such  as  the  simulated  erogenous  zones  depicted 
over  an  avatar.  Gaze  radial  angular  deviation 
(GRAD)  from  a  VMP  is  given  by  the  combination 
of  the  6  DOF  developed  by  head-movements 
and  the  2  DOF  (x  and  y  coordinates)  rendered 
by  the  eye-tracking  system^^'^^. 

While  variations  in  the  6  DOF  developed  by 
head-movements  define  momentary  changes  in 
the  global  scene  presented  through  the  HMD, 
the  2  DOF  given  by  the  eye-tracking  device  allow 
the  computation  of  the  line  of  sight’s  positioning 


relative  to  VMPs  (Figure  1  and  3).  The  closer 
this  measure  gets  to  zero,  the  closer  the  gaze 
dwells  in  the  vicinity  of  the  selected  VMP.  More¬ 
over,  VMPs  are  locked  onto  virtual  objects  and 
move  jointly  with  them  which  allows  analysis  of 
visual  pursuit  with  dynamic  virtual  objects. 

As  other  measurements  ensuing  from  this 
method,  the  subject’s  distance  from  the  VMPs 
(which  is  a  reliable  index  of  social  proxemics; 
figure  5),  the  pupil  size  diameter  (which  may  be 
indicative  of  preferences)  and  the  blinking  re¬ 
sponse  (which  is  a  good  index  of  a  defensive 
state)  are  also  obtainable^^'^®. 

These  oculomotor  measures  are  sampled  in 
synchronicity  with  the  erectile  response  which  is 
recorded  from  a  Limestone  technology  penile 
plethysmograph.  The  erectile  response  in  ques¬ 
tion  is  expressed  as  the  penile  circumference 
recorded  from  the  stretching  a  strain  gauge 
placed  around  the  penis  of  the  assessee  while 
he  is  in  immersion  with  a  sexual  avatar  (Figures 
1  and  4). 

Another  version  of  this  system  works  with  a 
standard  computer  flat  screen  combined  with  a 
pan-tilt  mobile  infrared  camera  installed  below. 
This  mobile  infrared  camera,  fine-tuned  by  a 
head-tracking  system  (IS-900  from  Intersense), 
captures  the  assessed  individual’s  eye  move¬ 
ments.  However,  with  this  particular  version  of 
the  system,  the  assessee’s  inherent  tridimen- 
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Figure  2  Ocular  saccades  (lines)  and  fixations  (dots)  patterns  as  superimposed  over  a  male  adult  ava¬ 
tar  (right)  and  a  female  adult  avatar  (left);  2  minutes  data  sampling  coming  from  one  male  heterosexual 
subject. 


sional  visual  perspective  is  not  grasped  as  such 
like  with  full  immersive  video-oculography  as 
described  above.  With  this  pan-tilt  mobile  infra¬ 
red  camera  method,  measurements  are  limited 
to  the  recording  of  eye  movements  as  ex¬ 
pressed  by  ocular  saccades  and  fixations 
mapped  over  a  2D  plane.  Saccades  are  the 
most  common  eye  movements,  they  involve 
rapid  jumps  of  the  eyes  from  one  position  to 
another  while  fixation,  i.e.  the  steady  gaze  hold¬ 
ing  over  stationary  objects,  can  be  considered 
as  a  special  case  of  visual  pursuit  in  which  the 
target  is  at  zero  velocity:  fixation  time  ranges 
from  150ms  to  600ms  with  an  average  of 
300ms;  jumps  observed  in  major  saccadic  eye 
movements  are  usually  larger  than  1.2  de¬ 
gree^®.  Figure  2  presents  two  examples  of  ocu¬ 
lar  saccades  (lines)  and  fixations  (dots)  pat¬ 
terns  as  superimposed  over  a  male  adult  avatar 
(right)  and  a  female  adult  avatar  (left). 


The  virtual  stimuli  that  we  use  to  prompt  sexual 
attraction  and  arousal  are  animated  naked  ava¬ 
tars.  These  have  been  designed,  developed  and 
validated  in  order  to  make  sure  that  they  would 
be  perceived  as  representing  the  required  sexual 
properties  to  assess  sexual  preference  with  pe¬ 
dophile  patients^^'^®.  Other  paraphilias  will  of 
course  require  the  development  of  other  specific 
virtual  environments  and  avatars.  See  section  3 
for  statistical  results. 

You  can  get  access  to  online  video  and  Power 
point  documents  presenting  this  method  as  well 
as  the  avatars  that  we  developed  (http:// 
w3.uqo.ca/cvberpsv/atsa2004vid.avi:  http:// 

w3.uqo.ca/cvberp$v/atsa2004pdf.pdf: ''®. 
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Figure  3.  Gazing  response  as  expressed  in  angular  deviation  from  a  virtual  measurement  point 
(VMP)  marking  out  the  avatar’s  head;  2  minutes  data  sampling  coming  from  one  subject  (expressed 
in  degree). 
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Figure  4.  Erectile  response  as  expressed  in  millimeters  of  penile  gauge  stretching,  in  function  of 
time;  2  minutes  data  sampling  coming  from  one  subject. 


ASSESSMENT  OF  THE  AVATARS’  PER¬ 
CEIVED  AGE  AND  LEVEL  OF  REALISM 

As  mentioned  in  section  2,  the  avatars  the  we 
use  have  been  designed,  developed  and  vali¬ 
dated  in  order  to  make  sure  that  they  would  be 
perceived  as  representing  the  required  sexual 
properties  and  the  required  age  to  assess  sex¬ 
ual  preference  with  pedophile  patients.  We  also 
developed  a  sexually  neutral  avatar  for  control 
purposes;  all  avatars  are  animated  in  a  similar 
fashion,  that  is  following  the  same  general  way 
but  with  slight  adaptations  to  fit  each  age  period 
and  gender  behavioural  properties.  Figure  5 
presents  snapshot  images  of  the  seven  catego¬ 
ries  of  avatars,  male  and  female  adults,  male 
and  female  adolescents  and  male  and  female 
children,  and  the  sexually  neutral  avatar.  These 
were  developed  to  respectively  emulate  the 


following  age  periods :  between  25  and  35 
years  old  for  the  adult,  between  13  and  17 
years  old  for  the  adolescent,  and  between  6 
and  10  years  old  for  the  children.  All  prototypes 
(except  the  sexually  neutral)  were  presented  to 
a  sample  a  63  subjects,  20  males  and  43  fe¬ 
males,  that  had  to  estimate  their  age  and  level 
of  realism  (Question:  On  a  scale  ranging  from  1 
to  7,  to  what  extent  do  you  consider  this  char¬ 
acter  realistic?).  Descriptive  statistics  for  the 
age  and  level  of  realism  results  are  displayed 
respectively  in  table  1  and  2. 

Repeated  measures  analyses  of  variance  and 
pairwise  comparisons  confirmed  that  avatars 
are  perceived  as  representing  members  of  dis¬ 
tinct  age  groups  (adults,  adolescents  and 
children),  with  slight  differences  between  pairs 
in  each  group  (F(5,58)=383.27,  p<.0001):  for 


Distance  fluctuations  from  the  avatars'  head  (ft) 


Figure  5.  Measures  of  social  proxemics;  distance  fluctuations  from  one  of  the  VMP  edited  over  the 
avatar’s  head;  2  minutes  data  sampling  coming  from  one  subject  (expressed  in  feet). 
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Figure  5.  Snapshot  images  of  the  avatars'  prototypes:  male  and  female  adults,  male  and  female 
adolescents,  male  and  female  children  and  a  sexually  neutral  avatar  for  control  purposes. 


Instance,  the  male  adult  avatar  Is  perceived  as 
1.9  years  older  than  the  adult  female  avatar 
and  the  male  child  avatar  is  perceived  as  0.635 
year  older  than  the  female  child  avatar.  The 
same  kind  of  analyses  also  confirmed  that  the 
overall  level  of  realism  is  acceptable  and  would 
need  improvements  (F(5,58)=16.251,  p<.0001). 
We  also  found  differences  between  categories: 
the  adolescent  avatars  are  perceived  as  the 
least  realistic  of  all,  and  even  more  so  for  the 
female  adolescent  avatar,  the  male  adult  avatar 
is  perceived  as  slightly  more  realistic  than  the 
female  adult  avatar  and  the  children  avatars  are 
perceived  as  the  most  realistic,  tied  with  the 
male  adult  avatar.  These  results  will  guide  fur¬ 
ther  improvements  of  these  stimuli. 


APPLICATIONS 

First,  the  method  put  forward  in  this  paper  is 
about  controlling  the  attentional  content  of  the 
assessee  through  the  minute  examination  of  his 
observational  behavior.  To  know  if  the  as- 
sessee’s  eyes  are  open  or  not  is  the  very  first 
requisite  that  our  method  answers  to.  To  pin¬ 
point  the  gaze  location  relatively  to  the  morpho¬ 
logical  layout  of  sexual  avatars  is  a  second  cru¬ 
cial  aspect  of  the  method  that  can  significantly 
increase  the  internal  validity  of  the  sexual  pref¬ 
erence  assessment  procedure  based  on  penile 
plethysmography.  By  knowing  how  and  when 
the  assessee  avoids  visual  contact  with  the 
avatar’s  features  being  able  to  arouse  his  sex¬ 
ual  interest,  approach  and  genital  responses. 


Avatars  (gender 
and  age): 

Perceived  age 

Min  value 

Max 

value 

Average 

SD 

Age  Male  adult 

15.00 

40.00 

26.5714 

5.27842 

Age  Female  adult 

16.00 

36.00 

24.6508 

3.98853 

Age  Male  adoles¬ 
cent 

10.00 

30.00 

14.5397 

3.35446 

Age  Female  ado¬ 
lescent 

9.00 

40.00 

14.4286 

4.47831 

Age  Male  Child 

4.00 

9.00 

6.8413 

1.23401 

Age  Female  Child 

4.00 

9.00 

6.2063 

1 .23339 

Table  1.  Descriptive  statistics  for  the  avatars  perceived  age  (N=63). 
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Avatars  (gender 
and  age): 

Min  value 

Max 

value 

Average 

SD 

M  adult 

3.00 

7.00 

4.7619 

1.16001 

F  adult 

2.00 

7.00 

4.5238 

1.30578 

M  adolescent 

2.00 

7.00 

4.1746 

1.39746 

F  adolescent 

1.00 

6.00 

3.5873 

1.32756 

M  child 

2.00 

7.00 

4.8889 

1.24578 

F  child 

2.00 

7.00 

4.8730 

1.15692 

Table  2.  Descriptive  statistics  for  the  avatars  estimated  level  of  realism  on  a  scale  ranging  from  1  to 
7  (N=63). 


the  internal  validity  of  the  erectile  response 
measurement  is  indeed  boosted. 

Secondly,  this  method  is  also  about  the  possi¬ 
bility  to  develop  an  original  index  of  sexual  pref¬ 
erence  whose  basis  would  be  visual  scanpath’s 
parameters  in  themselves.  Preliminary  results 
tend  to  incline  toward  the  relative  time  spent  on 
and  off  the  sexually  prompting  areas  displayed 
on  the  avatars  (as  tagged  by  the  VMPs),  as  well 
as  the  gaze  moving  sequence  from  one  area  to 
another,  as  good  candidates  to  diagnose  prop¬ 
erly  sexual  attraction,  deviant  or  nof^'^®.  The 
inherently  dynamic  properties  of  the  oculomotor 
signal,  that  is  how  this  latter  fluctuates  in  time, 
its  velocity,  acceleration  or  dynamic  properties 
of  higher  level,  could  also  be  indicative  of  sex¬ 
ual  interests^^'^^'^®.  The  latter  measurement  po¬ 
tentials  could  be  combined  as  well  with  meas¬ 
ures  of  proxemics,  pupillometrics  and  ocular 
defensive  response  to  yield  a  composite  index 
of  sexual  preference. 

Mediated  biofeedback  and  operant  conditioning 
of  sexual  behavior  and  perception 

Sexual  deviation  is  treated  in  various  ways  but 
mainstream  therapies  for  paraphilias  rely  es¬ 
sentially  on  cognitive-behavioral  techniques 
notably  with  the  use  of  aversive  therapy  and 
covert  sensitization  to  alter  deviant  sexual 
arousal,  behavior  and  interest®.  Penile  biofeed¬ 
back,  with  and  without  signaled  punishment, 
has  also  been  tried  with  interesting  results  to 
help  sex  offenders  gain  control  over  their  devi¬ 
ant  behaviors^®'®. 


Classically,  feedback  stimuli  in  biofeedback 
have  been  mostly  audio  or  visual  signals  sent 
continuously  and  monotonously  in  proportion  to 
physiological  parameters  (e.g.  amplitude  and 
frequency®'^^).  More  recently  however,  blofeed- 
back  applications  have  been  used  through 
video  games  and  virtual  reality  set-ups,  opening 
new  windows  on  more  complexly  mediated  in¬ 
teractions  between  patients  and  their  to  be  con¬ 
trolled  physiological  responses.  In  the  same 
vein,  we  developed  a  mediated  oculomotor  bio¬ 
feedback  prototype  that  allows  the  interacting 
subject  to  be  immersed  with  virtual  objects 
whose  kinematical  and  textural  properties 
change  in  real-time  in  function  of  gazing  re¬ 
sponses  and  pupillometric  inputs^®. 

We  propose  here  a  device  to  deliver  specifically 
a  combined  penile/oculomotor  biofeedback  in 
order  to  help  paraphiliacs  be  attuned  to  their 
physiological,  behavioral  and  perceptual  proc¬ 
esses  othenwise  not  directly  perceptible  to  them 
and  that  engage  their  sexuality  into  reprehensi¬ 
ble  actions.  This  biofeedback  device  would  me¬ 
diate  its  feedbacks  through  virtual  objects’  be¬ 
haviors.  For  instance,  a  patient  showing  signs 
of  interest  and/or  sexual  arousal  when  doing  a 
virtual  stroll  nearby  a  simulated  schoolyard 
could  be  continuously  kept  in  resonance  with 
his  physiological  reactions  by  the  delivering  of 
proportional  changes  in  the  ambiant  sound  and 
luminosity  of  the  scene  in  which  he  would  be 
immersed. 

As  for  aversive  conditioning,  the  coupling  of 
aversive  or  sensitizing  contingencies  with  sex- 
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ual  responses  could  be  done  automatically,  in  a 
discrete  fashion,  when  a  preset  criterion  would 
be  reached.  For  instance,  a  patient  exceeding  a 
preset  criterion  equivalent  to  10%  of  his  maxi¬ 
mum  erectile  response  and  displaying  at  the 
same  time  a  critical  visual  sampling  pattern  with 
regard  to  potential  simulated  victims  would  see 
himself  provided  with  an  aversive  or  sensitizing 
stimulus  that  could  take  the  form  of  a  virtual 
policeman  entering  the  precinct;  alternatively, 
the  stimulus  could  be  purely  and  simply  a  dras¬ 
tic  removal  from  the  scene  to  an  other  environ¬ 
ment  designed  to  be  aversive,  eg:  a  virtual  cell 
or  a  virtual  arrest  situation  or  any  event  that 
would  be  particularly  aversive  to  that  individual. 

CONCLUSION 

Concrete  applications  of  virtual  reality  in  mental 
health  are  becoming  more  and  more  common. 
However,  virtual  reality  applications  for  forensic 
science  and  correctional  rehabilitation  are  still 
not  very  numerous.  With  the  method  put  for¬ 
ward  in  this  paper  we  tried  to  bring  solutions  to 
real  and  important  psychological  and  social 
problems  by  using  the  immersive  properties  of 
virtual  reality  together  with  the  analytical  poten¬ 
tial  of  eye-tracking  technologies. 
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Abstract:  Specific  phobias  are  one  of  the  most  frequent  mentai  health  problems  and  can  lead  to 
years  of  personal  suffering.  The  most  effective  treatment  is  exposure  therapy.  Our  aim  was  to  proof 
the  feasibility  and  efficacy  of  virtual  environments  in  treating  acrophobia  patients  using  a  manually 
guided  exposure  therapy.  Our  pilot  study  was  designed  as  a  crossover  intervention  with  a  waiting  list 
condition  as  a  control  group.  After  treatment,  our  results  show  that  exposure  in  virtual  environments 
is  a  feasible  technique  can  provoke  anxiety,  and  leads  to  a  therapeutic  effect. 


INTRODUCTION 

Anxiety  disorders  appear  with  a  lifetime  preva¬ 
lence  of  about  25%  (12).  Chronic  processes 
have  been  observed  up  to  a  percentage  of  50% 
in  agoraphobic,  specifically  phobic  and  socio¬ 
phobic  cases  (24).  Among  these,  specific  pho¬ 
bias  are  the  most  widespread  disorders  (11, 
16).  Currently,  the  internationally  accepted 
treatment  strategy  is  behavior  therapy  with  ex¬ 
posure  (20,  21).  Many  trials  have  been  con¬ 
ducted  to  substitute  situations  of  exposure  us¬ 
ing  computer-based  simulations.  Today  one 
can  say  that  the  attempts  to  develop  computer- 
based  psychotherapy  programs  have  not  been 
very  successful. 

But,  the  early  1990s  saw  a  change,  brought 
about  by  the  introduction  of  virtual  reality  (VR) 
systems  in  research  at  universities.  Three- 
dimensional  virtual  environments  are  computer¬ 
generated  interactive  visualizations,  in  which 
the  user  is  able  to  experience  spatial  and  tem¬ 
poral  correlations.  Data  helmets  or  specially 
designed  presentation  media,  based  on  func¬ 
tion  units  composed  of  screens  and  appropriate 
projectors,  are  used  to  produce  the  three- 
dimensional  effects.  In  the  last  years,  virtual 
environments  have  often  been  used  by  several 
research  teams  in  Europe  and  the  USA,  to  re¬ 
search,  for  example,  acrophobia  (3,  8),  claus¬ 
trophobia  (4),  arachnophobia  (5)  and  fear  of 
flying  (9,26,27,28).  This  technique  is  also  used 
to  study  addictions  (cue  exposure)  (15).  Fur¬ 
thermore,  expectations  are  raised  that  by  apply¬ 


ing  virtual  environments  as  an  exposure  plat¬ 
form  there  will  also  be  a  great  success  in  the 
investigation  of  the  biological  basis  of  specific 
phobias;  Because  the  environments  generated 
by  the  computer  are  always  represented  com¬ 
pletely  unmodified  and  exactly  corresponding  to 
the  preceding  and  succeeding  sequences,  they 
are  very  suitable  for  studying  the  psycho- 
physiological  stimulus-reaction  conditions  and 
for  examining  the  therapeutic  effects  on  these 
psycho-physiological  reactions  (23). 

PILOT  STUDY 

METHODS 

Study  Design 

At  first,  three  test  persons  were  exposed  to  aiti- 
tude  in  VR  while  the  other  three  test  persons 
were  supervised  in  a  waiting  list  condition 
(WLC).  V^C  means  that  the  test  persons  could 
perform  self-selected  computer  games  concur¬ 
rently  to  the  exposure  therapy.  After  a  break  of 
three  months  the  test  persons  in  the  WLC  did 
the  VR  exposure,  and  the  group  that  had  al¬ 
ready  been  exposed  to  altitude  played  on  the 
computer. 

In  a  preliminary  talk  the  therapeutic  rationale 
was  discussed  with  the  test  persons.  All  of  the 
six  treatments  (each  with  three  exposures) 
were  carried  out  by  a  therapist  with  many  years 
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of  clinical  experience  in  psychiatry  and  training 
in  cognitive  behavior  therapy.  On  the  basis  of 
empirical  and  theoretical  models  the  origin,  trig¬ 
gering  factors  and  the  perpetuating  factors  of 
the  specific  phobia  were  explained.  Also  the 
correlation  of  cognitive,  affective  and  vegetative 
symptoms  of  anxiety  with  situational  character¬ 
istics  was  described.  The  model  of  a  dysfunc¬ 
tional  interaction  (also  known  as  “vicious  circle”) 
of  intrapersonal  and  external  stimuli  was  well 
understood  by  the  concerned  people  because 
of  their  own  experience.  Besides,  the  negative 
impact  of  anxiety-avoidance  on  the  process  of 
the  disorder  was  emphasized.  The  program 
sequence  and  the  expectations  of  the  test  per¬ 
son  were  clarified.  The  expectations  were,  at 
first,  very  skeptical  and  affected  by  frustration  of 
many  years,  but  after  the  medical  education  the 
attitude  changed  regularly  into  a  rather  hopeful 
one.  The  test  persons  were  encouraged  to  fre¬ 
quent  autonomously,  in  the  period  between  the 
sessions,  situations  which  they  had  avoided  so 
far.  All  test  persons  were  physically  examined 
by  a  medical  doctor.  Therefore,  somatic  dis¬ 
eases  sometimes  related  to  anxiety  syndromes 
could  be  excluded,  in  doing  so,  a  psychiatric 
and  medical  anamnesis  was  made. 

The  diagnosis  of  a  specific  phobia  (acrophobia) 
was  made  by  the  Composite  International  Diag¬ 
nostic  Interview  (CIDI  (25)).  Other  phobias  or 
comorbid  disorders  according  to  ICD-10  and 


DSM  IV  (1)  were  excluded.  Especially  any  psy¬ 
chotic  disorder,  suicide  attempts  in  the  anamne¬ 
sis,  disorders  caused  by  using  psychotropic 
substances  like  addiction,  delirium  and  appar¬ 
ent  physical  disorders  (especially  considerably 
impaired  vision  or  defective  hearing)  were  crite¬ 
ria  for  exclusion. 

The  therapeutic  effect  was  measured  by  means 
of  self-evaluation  on  a  4-point-scale  at  5  points 
in  time:  4  of  them  were  before  and  after  expo¬ 
sure/waiting  list  (respectively  waiting  list/ 
exposure)  and  one  interview  was  conducted  6 
months  later  (figure  1a).  We  asked  the  test  per¬ 
sons  about  the  subjective  evaluation  of  their 
discomfort  (scale  “comfort/discomfort”)  and  their 
contentment  (scale  “contentment”)  and  also 
about  the  dimension  of  phobic  avoidance  (scale 
“avoidance”).  These  3  scales  measured  from  “I 
feel  very  good”  (point  1)  to  “I  feel  very 
bad”  (point  4). 

Furthermore,  the  concept  of  the  Subjective 
Units  of  Discomfort  (SUD  (7))  was  explained  to 
the  test  persons.  It  refers  to  an  individually 
“calibrated”  rating  of  the  actual  anxiety  and 
stress  on  a  scale  of  0-100.  During  the  later  ex¬ 
posure,  the  test  person  was  asked,  in  intervals 
of  5  min,  about  the  level  of  his  anxiety/stress  on 
a  scale  of  0  (conveniently  balanced  feeling)  to 
100  (maximal  anxiety).  The  purpose  of  that 
was,  on  the  one  hand,  to  identify  potential  cog¬ 
nitive  avoidance-strategies  and  on  the  other 
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Figure  1.  First  exposure  treatment  of  test  person  No.  5.  The  horizontal  axis  shows  39  intervals  of  5 
minutes.  The  left  vertical  axis  indicates  the  dimension  of  anxiety  (black  line  with  squares)  between  0 
and  100,  the  right  vertical  axis  indicates  the  altitude  of  the  elevator  in  meters  (maximal  220m, 
dashed  line  with  lozenges) 
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hand  to  survey  the  impact  of  the  virtual  environ¬ 
ment  on  the  stress  of  the  test  persons.  Further¬ 
more,  the  test  persons  were  asked  to  give  their 
subjective  evaluation  regarding  the  dimension 
of  the  “worst”  anxiety  situation.  The  scale 
“maximal  anxiety”  was  divided  in  10  stages  to 
simplify  an  adaptation  to  the  100  scale  points  of 
the  SUDs. 

The  Virtual  Environment 

The  computer-based  program  is  generated  by 
True  Space,  TCL  (the  software  can  be  bought) 
and  by  Lightning  (virtual  interaction-  and  anima¬ 
tion  software  especially  created  for  our  applica¬ 
tion,  ICIDO  Ltd.,  Stuttgart,  Deutschland).  The 
virtual  environment  is  visualized  by  a  Silicon 
Graphics  (SGI)  Onyx^  Deskside  System.  For 
data  transfer  a  Head  Mounted  Display  (V8,  Vir¬ 
tual  Research),  a  Tracking  System  (Flock  of 
Birds,  Ascension)  and  an  interaction  medium 
(Space  Mike,  ICIDO  Ltd.),  also  especially  cre¬ 
ated  for  our  application,  were  used. 

The  Exposure 

The  gradual  exposure  in  VR  was  made  by 
means  of  the  aforementioned  technique,  alti¬ 
tude  “situations”  generated  by  the  computer 
served  to  elicit  anxiety.  A  manual  was  used  and 
(13)  served  to  assure  standardization  of  the 
therapy  process;  the  virtual  environment  was 
always  generated  homogenously  by  the  com¬ 
puter. 

The  manual  consisted  of  an  introduction  to  the 
technique  and  information  about  specific  pho¬ 
bias  as  well  as  a  therapy  schedule. 

•  In  session  1 ,  in  addition  to  the  diagnosis  via 
the  CIDI  and  the  medical  examination,  a  behav¬ 
ioral  analyses  (problem  definition,  analyses  of 
condition  and  function,  developing  an  individual 
phobia  hierarchy)  was  made.  The  identification 
of  further  problem  areas  completed  the  data 
acquisition. 

•  In  the  second  session  the  therapy  rationale, 
the  schedule  and  the  expectations  were  dis¬ 
cussed.  The  test  person  received  instructions 
for  practicing  in-between  the  sessions. 

•  In  session  3  to  5,  the  test  person  was  con¬ 
fronted  with  the  virtual  environment. 

In  a  final  session  the  experience  of  the  test  per¬ 


son  made  during  the  therapy  was  discussed. 
The  person  was  again  encouraged  to  continue 
confronting  himself  with  situations  he  had 
avoided  so  far. 

The  first  of  three  confrontations  was  dominated 
by  the  cognition  of  anxiety.  The  test  person 
should,  if  possible,  experience  maximal  occur¬ 
rence  of  anxiety  cognitions,  anxiety  feelings  and 
anxiety  physiology.  Every  session  lasted  2-4 
hours.  During  the  exposure  the  test  person  was 
asked  every  5  min  about  the  level  of  his  anxi¬ 
ety/stress  measured  on  a  scale  ranging  from  0- 
100  (SUDs).  After  having  surpassed  the  point  of 
culmination  of  the  emotional-vegetative  reac¬ 
tion,  the  exposure  continued  until  the  test  per¬ 
son  showed  a  significant  subjective  ease 
(scale:  <30). 

The  test  person  was  instructed  to  verbalize 
clearly  his  perception.  The  therapist  was  em- 
pathic-supporting  and  motivated  the  test  person 
to  endure  the  situation.  In  principle,  the  decision 
to  drop  out  or  to  continue  the  exposure  was  not 
made  by  the  therapist.  If  the  test  person  wanted 
to  drop  out,  the  therapist  was  supposed  to  en¬ 
courage  him  to  continue.  An  “emergency-stop 
procedure”  has  been  arranged  individually  (i.e. 
verbal  signs  as  “Stop,  it  is  enough”). 

Test  Persons 

The  test  persons  were  people  from  the  Ger¬ 
man-speaking  area  around  Basel.  There  was 
an  advertisement  in  the  newspaper  and  in  the 
period  from  February  2000  to  January  2001,  38 
persons  answered  the  advertisement.  Twenty 
of  them  could  be  excluded  already  on  the 
phone,  because  either  they  did  not  have  acro¬ 
phobia  or  they  had  some  other  symptoms  as 
i.e.  eating  disorders,  alcohol  addiction,  and  ago¬ 
raphobia.  Eighteen  people  were  invited  to  a 
preliminary  talk,  but  4  of  them  cancelled  it  be¬ 
forehand.  After  the  14  preliminary  talks,  2  per¬ 
sons  withdrew  their  consent  for  the  treatment.  6 
of  the  remained  12  persons  were  selected  via  a 
random  generator  and  were  accepted  for  the 
pilot  study. 

All  6  test  persons  were  male  and  did  not  suffer 
from  a  further  somatic  mental  disorder.  They 
were  socially  well-integrated  and  employed. 
The  mean  age  was  37.8  years. 
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Figure  2.  First  exposure  treatment  of  test  person  No.  6.  The  horizontal  axis  shows  39  intervals  of  5 
minutes.  The  left  vertical  axis  indicates  the  dimension  of  anxiety  (black  line  with  squares)  between  0 
and  100,  the  right  vertical  axis  indicates  the  altitude  of  the  elevator  in  meters  (maximal  220m, 
dashed  line  with  lozenges) 


RESULTS 

The  object  of  this  pilot  study  was  to  examine 
the  feasibility  and  the  efficacy  of  virtual  environ¬ 
ments  in  treating  acrophobia  patients.  The  ex¬ 
posure  was  without  any  relevant  incidents  and 
the  individually  arranged  “emergency-stop”  was 
never  used. 

Positive  effects  of  the  exposure  in  the  virtual 
environment  could  be  observed  for  all  test  per¬ 


sons.  Exemplary  results  of  the  18  exposures 
are  graphically  illustrated  in  figure  1,  2  and  3. 
Figure  1  represents  a  “classical”  exposure  proc¬ 
ess  within  105  min:  At  the  beginning  the  anxiety 
increased  synchronously  with  the  altitude.  In 
the  subsequent  2  confrontations,  the  amplitude 
of  the  subjectively  experienced  anxiety  de¬ 
creased  significantly  and  ends  after  105  min 
with  less  than  30  SUDs.  In  the  second  case 
(figure  2),  the  anxiety  was  already  at  the  begin¬ 
ning  very  distinct.  The  exposure  was  slow  and 


Figure  3.  First  exposure  treatment  of  test  person  No.  4.  The  horizontal  axis  shows  39  intervals  of  5 
minutes.  The  left  vertical  axis  indicates  the  dimension  of  anxiety  (black  line  with  squares)  between  0 
and  100,  the  right  vertical  axis  indicates  the  altitude  of  the  elevator  in  meters  (maximal  220m, 
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gradual  and  reached  only  after  approximately 
50  min  the  maximal  altitude.  The  anxiety  be¬ 
came  almost  insupportable  for  the  test  person 
and  a  reduction  of  altitude  implicated  almost  no 
release.  After  about  100  min  the  anxiety  dimin¬ 
ished  in  spite  of  subsequent  maximal  altitude- 
exposure.  The  fast  virtual  downward  trip  by  the 
elevator  resulted  firstly  in  an  increase  of  anxiety 
but  finally  in  a  great  release.  After  160  min  a 
reduction  of  anxiety  can  be  observed  ending 
after  almost  200  min  with  a  SLID  of  approxi¬ 


mately  30.  Figure  3  shows  a  prolonged  rise  of 
anxiety.  Only  at  the  second  time  of  maximal 
altitude  the  stress  level  had  a  relevant  value  of 
about  70  SUDs.  The  release  occurred  soon  and 
was  maintained  also  after  new  confrontation. 
After  145  min  the  exposure  was  stopped  having 
a  stable  SUDs  of  less  than  10. 

Figure  4  shows,  resuming  all  5  measurements, 
the  test  persons’  self-evaluation  concerning 
“contentment”,  “discomfort”,  “avoidance”  and 


Figure  4a-f.  Illustration  of  therapy  effects  concerning  “contentment”  [Zufriedenheit],  “comfort/ 
discomfort”  [Befindlichkeit],  “avoidance”  [Vermeidung]  and  “maximal  anxiety”  [maximale  Angst]:  a-c 
represents  test  person  no.  1-3;  d-f  represents  test  person  no.  4-6.  Time  of  enquiry:  measuring  1 
(black),  measuring  2  (diagonally  striped),  measuring  3  (dark  grey),  measuring  4  (medium  grey),  in- 
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maximal  anxiety  in  single  situations.  One  can 
see  that  all  test  persons  experienced  the  ther¬ 
apy  as  successful.  This  estimation  was  stable 
for  6  months:  The  subjectively  felt  comfort  in¬ 
creased  and  previously  avoided  situations  were 
frequented  more  and  more  and  were  experi¬ 
enced  with  decreasing  anxiety.  Discontent,  dis¬ 
comfort  and  the  degree  of  avoidance  were 
evaluated  by  the  test  persons  on  a  scale  rang¬ 
ing  from  0  (no  problem)  to  4  (every  altitude  with 
more  than  5  m  was  avoided).  The  evaluations 
were  to  be  explicitly  subjective  and  general.  In 
terms  of  “discontent”  the  test  person  was  asked 
to  what  extent  he  was  discontent  because  of  his 
phobia.  With  regard  to  “comfort/discomfort”  he 
was  asked  how  much  the  phobia  affects  his 
physical  and  mental  orientation.  An  occurrence 
of  2  and  3  improved  to  0-1 .  “Avoidance”  meant 
the  avoidance  of  being  in  altitude.  This  avoid¬ 
ance  was  at  the  beginning  3-4  and  after  6 
months  only  0-1 .  The  occurrence  of  anxiety  was 
also  quantified  by  a  0-100-scale.  All  test  per¬ 
sons  experienced  values  of  90  or  even  more 
before  they  underwent  the  therapy.  After  6 
months  the  Subjective  Units  of  Discomfort  were 
only  around  10. 

DISCUSSION 

The  pilot  study  shows  that  an  exposure  in  a 
virtual  environment  is  possible  and  evokes  anxi¬ 
ety  among  patients  suffering  from  a  specific 
phobia  and  has  therapeutic  effects.  It  has  not 
been  clarified  so  far  to  what  extent  habituation- 
processes  and  to  what  extent  cognitive  restruc¬ 
turings  took  place. 

Possible  side  effects  of  virtual  environments  are 
sickness  in  the  simulator  (2)  or  impaired  vision 
after  the  exposure  (18).  According  to  our  inquiry 
none  of  the  6  test  persons  experienced  such  or 
other  side  effects.  The  experience  was  domi¬ 
nated  by  anxiety  symptoms  as  i.e.  sweating, 
gastrospasms,  nervousness  and  sometimes 
even  “panic”  anxiety.  The  pressure  of  the  data 
helmet  on  the  head,  vertigo,  sickness  or  further 
disturbances  were  not  said  to  be  disagreeable. 
It  can  be  supposed  that  because  of  the  subjec¬ 
tive  threat  of  the  virtual  altitude  such  further  dis¬ 
agreeable  feeling  have  been  suppressed.  Com¬ 
pared  to  exposures  in  vivo,  the  test  persons 
showed  -in  vitro-  very  similar  reaction  patterns. 
It  is  remarkable  that  nobody  used  the  emer¬ 


gency-stop  even  if  they  often  talked  about  doing 
it.  In  a  real  exposure  such  moments  are  very 
critical  because  the  avoidance  is  easy  to  do. 
Drop-out  is  known  as  a  common  strategy  and  is 
almost  automated.  The  very  different  way  of  the 
virtual  environment  could  be  a  cesura  in  this 
automation.  These  experiences  have  to  be  in¬ 
terpreted  with  caution.  The  test  persons  were 
typical  for  specific  phobic  disorders  insofar  as 
the  beginning  of  the  disorder  was  in  early  adult¬ 
hood  and  was  chronically  restricting.  Among  all 
the  healing  attempts  there  were  psychothera¬ 
peutic  treatments  of  “school  psychology”,  so- 
called  alternative  intervention  and  even  esoteric 
practices.  Some  of  our  test  persons  also  tried 
sporadically  to  reduce  their  anxiety  in  taking 
“benzodiazepines”  or  alcohol. 

Because  of  using  self-evaluation-scales  there  is 
another  limitation  of  the  data.  This  process  is 
common  in  therapeutic  terms,  but  leads  to  diffi¬ 
culties  in  the  evaluation  of  the  data  and  in  its 
comparability.  The  technique  of  the  VR  could 
be  a  great  advantage  because  of  its  standardi¬ 
zation.  During  exposure,  additional  physiologi¬ 
cal  parameters  (23)  could  be  measured  forming 
the  objective  of  the  so  far  only  subjective  aspect 
of  anxiety.  In  this  case,  the  investigation  of  the 
biological  basis  of  the  specific  phobias  is  auspi¬ 
cious  by  means  of  using  virtual  environments 
as  an  exposure  platform.  Because  the  com¬ 
puter-generated  environments  are  always  rep¬ 
resented  completely  unmodified  compared  to 
the  preceding  and  succeeding  processes,  they 
are  suitable  for  the  study  of  the  psycho- 
physiological  stimulus-reaction  conditions  and 
for  the  examination  of  the  therapeutic  effects 
resulting  from  these  psycho-physiological  reac¬ 
tions. 

CONCLUSION 

Our  pilot  study  showed  an  improvement  in  all 
the  measured  domains  (contentment,  comfort/ 
discomfort,  anxiety  occurrence,  avoidance). 
The  occurrence  of  the  test  person’s  anxiety  di¬ 
minished  significantly  during  exposure.  Situa¬ 
tions  previously  avoided  were  frequented.  Gen¬ 
eral  life-contentment  and  comfort  rose.  The  im¬ 
provement  after  the  therapy  is  very  evident 
among  the  test  persons.  The  waiting  condition 
“computer  games”  changed  nothing.  After  6 
months  the  positive  results  could  still  be  ob- 
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served  and  they  even  continued  to  improve. 
These  results  correspond  to  our  own  therapeu¬ 
tic  experience  as  well  as  to  other  published 
studies  (8,  9,  10,  17,  19,  20,  22,  26,  27,  28). 
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Virtual  Environments  to  Address  Autistic  Social  Deficits 

C.  Y.  Trepagnier,  Ph.D.,  M.  M.  Sebrechts,  A.  Finkelmeyer,  M.  Coleman,  W.  Stewart, 

Jr.,  M.  Werner-Adler, 

The  Catholic  University  of  America,  Washington,  DC 


Abstract:  Background:  Autistic  Disorder  is  defined  by  social  and  communicative  impairments  and 
restricted,  narrow  interests.  Impaired  motivation  is  also  common.  Several  investigators  have  devel¬ 
oped  computer-based  and  virtual  environment  tools  to  address  various  issues  in  autism(1-5).  Four 
projects  at  CUA  address  the  social  impairment. 

Method/Tools:  Face  Processing.  The  prediction  that  autistic  face  processing  impairment  involves 
gaze  differences  was  tested  in  an  eye  tracking  study  using  a  VR  display.  A  replication  in  progress 
uses  a  monitor.  Early  Intervention.  A  virtual  environment,  with  kiddie-ride,  monitor  display  and  eye 
tracker,  is  being  tested.  The  goal  is  to  induce  young  children  with  autism  to  attend  to  faces.  Social 
Navigation.  A  Joystick-navigable,  first-person-perspective  shopping  mall  is  presented  on  a  monitor. 
To  locate  objects,  the  user  must  move  either  between  or  around  social  and  non-social  obstacles. 
Data  include  user  comments  and  path  records.  Training  in  Social  Conversation.  SIMmersion  LLC™ 
are  collaborating  with  us  to  develop  a  social  conversation  module  for  adults  with  Asperger’s  Disor¬ 
der.  A  virtual  character  remembers  the  conversation  and  responds  as  would  an  actual  interlocutor  to 
the  user’s  speech. 

Results:  Face  Processing.  Preliminary  results  (with  headset)  confirmed  gaze  differences(6).  The  rep¬ 
lication  is  in  progress.  Early  intervention.  Reliability-testing  is  being  completed.  Social  Navigation. 
Controls  describe  their  avatar’s  actions  in  the  first  person,  and  refrain  from  walking  between  con¬ 
versing  characters.  Data  collection  with  persons  with  autism  has  begun.  Training  in  Social  Conversa¬ 
tion.  Development  is  in  progress. 

Discussion:  The  projects  described  above  derive  from  the  hypothesis  that  failure  to  establish  spe¬ 
cies-typical  face  attention  and  processing  itself  undermines  cognitive  development(7).  Accordingly 
face  processing  is  targeted  in  young  children,  and  compensatory  training  is  addressed  with  older 
individuals.  The  challenge  is  to  make  the  technology  not  only  effective,  but  adequately  entertaining 
to  overcome  autistic  motivational  barriers. 

INTRODUCTION 

Autism  Spectrum  Disorder  (ASD)  is  character¬ 
ized  by  impairment  in  verbal  and  nonverbal 
communication  skiils  and  social  interaction,  and 
the  presence  of  repetitive  behavior  and  narrow, 
obsessive  interests,  according  to  the  standard 
diagnostic  criteria  used  in  research  over  the 
past  decade.^  While  ASD  can  be  diagnosed  as 
early  as  18  months,  and  most  individuals  are 
diagnosed  by  or  near  the  start  of  their  educa¬ 
tional  career,  some  are  not  identified  until  later 
childhood  or  even  adulthood. 

Facial  expressions  and  gestures  often  hold  little 
meaning  for  individuais  with  ASD.  As  a  result  it 
can  be  very  difficult  for  them  to  make  sense  of 


people's  behavior.  During  adolescence  and 
young  adulthood  some  individuals  with  ASD 
develop  enough  insight  to  become  aware  that 
they  are  different  from  their  peers,  and  experi¬ 
ence  ioneliness  and  sadness. 

Once  thought  to  be  rare,  ASD  and  related  dis¬ 
orders  are  now  being  identified  in  large  num¬ 
bers.  Prevalence  estimates  range  from  15  to  60 
per  10,000  individuals.^'^  The  symptoms  of  au¬ 
tism  can  be  mitigated  and  quality  of  life  im¬ 
proved  thanks  to  evidence-based  treatments 
such  as  intensive,  autism-specific  intervention 
in  early  childhood, and  psychopharmacologi- 
cal  interventions.^'®  Despite  the  popular  percep- 
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tion  of  autism  as  a  disorder  of  childhood,  most 
individuals  with  ASD  are  adults. 

In  summary,  there  is  a  large  population  of  chil¬ 
dren  and  older  individuals  who  have  severe 
deficits  in  their  ability  to  interpret  facial  expres¬ 
sions  and  other  nonverbal  behavior;  and 
equivalent  lacks  in  their  ability  to  transmit  com¬ 
munication  through  these  channels.  The  ASD 
syndrome  results  in  a  variety  of  social  impair¬ 
ments  that  constitute  major  obstacles  to  the 
individual’s  possibilities  of  achieving  independ¬ 
ence  and  sustained  employment,  and  having 
friends  and  intimate  relationships. 

Interventions  to  modify  the  young  child’s  devel¬ 
opmental  course,  or  to  teach  older  individuals 
skills  that  were  not  acquired  naturally  in  child¬ 
hood  cannot  approach  in  duration  or  intensity 
the  on-going,  unavoidable  social  instruction  that 
is  experienced  by  typically  developing  individu¬ 
als.  The  most  intensive  autism  intervention  pro¬ 
grams  are  offered  primarily  to  young  children 
(generally  aged  two  to  four),  and  these  pro¬ 
grams  target  a  range  of  skills,  among  which 
social  interaction  is  not  an  important  focus.  In¬ 
tensive  programs  in  social  skill  acquisition  for 
older  children  are  rare;  for  adolescents  and 
adults  with  ASD  they  are  iargely  unavailable, 
and  the  majority  of  such  programs  are  not 
based  on  scientific  evidence. 

Virtual  environments  have  been  shown  to  be 
clinically  useful  for  distraction  from  pain®  ®  and 
in  the  treatment  of  phobias^®’’®  and  are  being 
investigated  for  their  applicability  to  a  number  of 
other  medical  and  psychiatric  problems.  Sev¬ 
eral  investigators  have  developed  computer- 
based  and  virtual  environment  tools  to  examine 
social  behavior  in  autism^'*'^®  and  to  improve 
face  processing,  emotion  recognition,  and  so¬ 
cial  problem  solving. The  Autism  Research 
Group  at  The  Catholic  University  of  America 
shares  this  focus  on  exploiting  computer  and 
virtual  reality  technology  to  develop  techniques 
to  improve  the  social  competence  of  individuals 
with  autism. 

Our  motivation  for  approaching  autism  mitiga¬ 
tion  through  the  social  deficits  grew  out  of  the 
recognition  of  the  centrality  of  the  social  deficit 
in  autism,^®  and  observation  and  review  of  face 
processing  impairments  and  presence  of  symp¬ 


toms  of  anxiety  and  depression  in  individuals 
with  ASD,®^  findings  that  continue  to  be  con¬ 
firmed  in  more  recent  studies  (e.g.,  face  proc¬ 
essing  impairment^^'^®;  psychiatric  comorbid- 
it/®  ®>  It  was  hypothesized  that  infant  onset  of 
symptoms  of  anxiety  and  depressive  disorder 
could  disrupt  face  attention,  with  the  conse¬ 
quence  that  the  bases  of  face  processing  would 
not  be  established  during  the  period  when  the 
infant  is  most  apt,  biologically,  neurologically 
and  behaviorally,  for  face  processing  to  take 
root.®®  Following  is  a  brief  description  of  four 
projects  at  CUA  that  address  autistic  social  im¬ 
pairments:  Face  Gace,  Early  Intervention,  the 
Virtuai  Mall,  and  development  of  a  Simulation  to 
Train  Conversational  Interaction 

FACE  GAZE 

These  studies  addressed  the  hypothesis  that 
autistic  differences  in  face  processing  would 
have  correlates  in  gaze  behavior,  specifically 
that  people  with  ASD  would  not  look  at  the 
same  locations  on  the  face  as  controls.  By 
means  of  a  VR  headset  in  which  an  eye  track¬ 
ing  camera  was  installed  significant  differences 
were  found  between  groups;  while  typically  de¬ 
veloping  participants  looked  at  the  interior  of  the 
face,  persons  with  ASD  showed  a  greater  ten¬ 
dency  to  look  at  the  periphery.®®  Since  use  of 
the  headset  excluded  many  potential  partici¬ 
pants,  a  replication  using  a  monitor  and  desk¬ 
top  eye  tracker  was  next  undertaken.  Prelimi¬ 
nary  analysis  of  these  data  has  found  a  signifi¬ 
cant  correiation  between  gaze  at  the  eye  area 
and  lesser  impairment  on  the  sociai  component 
of  the  Autism  Diagnostic  Interview-  Revised 
(ADI-R),®°  an  instrument  that  focuses  primarily 
on  the  individual’s  behavior  at  age  four.  This 
finding  suggests  that  deficits  in  early  social  de¬ 
velopment  may  have  a  lasting  effect  on  how 
these  individuals  look  at  faces.  Since  foveal  and 
perifoveal  vision  are  restricted  to  a  narrow  an¬ 
gle,  gaze  directed  at  the  facial  periphery®®  or  at 
the  mouth®®  fails  to  take  in  with  adequate  acuity 
the  information  necessary  for  interpreting  facial 
expression. 

EARLY  INTERVENTION 

An  outgrowth  of  this  project  has  been  to  pro¬ 
pose  an  intervention  to  teach  young  children 
with  autism  to  direct  their  gaze  to  the  eye  area 
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of  the  face,  and  to  convey  that  attention  to  the 
eye  area  has  inherent  functionai  benefits.  The 
goal  of  this  project,  therefore,  is  to  develop  and 
pilot  a  technique  to  induce  children  with  ASD 
aged  24  to  54  months  to  attend  to  the  meaning¬ 
ful  areas  of  faces,  and  to  make  use  of  the  infor¬ 
mation  transmitted  by  them.  In  order  to  accom¬ 
plish  this  it  is  critical  to  attract  and  sustain  the 
child’s  participation,  in  addition  to  differentially 
rewarding  the  target  gaze  behaviors,  as  sensed 
by  a  desktop  eye  tracker.  The  rewards  available 
for  this  purpose  include  a  wide  range  of  videos, 
and  ‘rides’  in  the  ’kiddie’  helicopter  in  which  the 
monitor  and  tracker  are  instalied.  Current  test¬ 
ing  addresses  the  validity  of  our  ‘semi¬ 
automatic’  calibration  technique  for  children, 
and  adjustments  to  the  positioning  of  a  child 
car-seat  and  head-rest  for  head  stabilization 
adequate  for  reliable  eye  tracking.  A  pre-pilot 
study  beginning  in  May  wiii  assess  the  feasibil¬ 
ity  of  the  training  schedule  for  children  and  their 
families. 

VIRTUAL  MALL 

Young  children  with  autism  may  exhibit  a  vari¬ 
ety  of  reactions  to  finding  themseives  in  large, 
busy  environments  such  as  airports  and  shop¬ 
ping  malls.  Among  the  reactions  particularly 
disconcerting  for  parents  can  be  an  apparent 
insensitivity  to  the  presence  of  other  peopie, 
treating  them  as  miidly  inconvenient  obstacles 
that  will  give  way  on  approach  or  even  contact. 
As  adolescents  and  adults,  some  people  with 
ASD  still  seem  unaware  of  social  spatial  con¬ 
ventions,  walking  between  two  people  engaged 
in  conversation  with  each  other,  or  passing  be¬ 
tween  a  person  and  a  display  window  at  which 
she  is  looking.  The  Virtual  Mall  is  being  devel¬ 
oped  with  the  participation  of  non-disabled  and 
ASD  adults  through  an  iterative  design  process, 
using  standardized  tasks  to  draw  the  user  to 
explore  the  environment  and  encounter  its  so¬ 
cial  navigational  challenges.  The  long-term  goal 
is  the  development  and  evaluation  of  a  clinically 
practical  and  ecologically  valid  assessment  of 
social  cognition,  and  a  rehabilitation  modality  to 
improve  the  ability  of  individuals  with  ASD  to 
function  in  everyday  environments. 

Data  has  been  compiled  from  8  participants’ 
comments  during  and  after  performance  of  the 
tasks,  as  well  as  an  automatically  acquired  re¬ 


cord  of  their  navigation  paths.  Participants  to 
date  have  been  4  females  and  4  males  (one 
with  an  ASD  diagnosis),  7  of  whom  were  Cau¬ 
casian  and  one  African-American,  ranging  in 
age  from  19-27  (Mean  age  20.9). 

Virtual  activities  were  defined  in  detail  for  the 
human  factors  trials  in  order  to  assure  that  all 
participants  experienced  all  of  the  challenges  of 
the  mall,  including  virtual  humans  and  inani¬ 
mate  objects,  situated  at  locations  that  require 
participants  to  navigate  through  or  around  them 
in  order  to  reach  their  target.  Placement  and 
orientation  of  the  virtual  humans  creates  situa¬ 
tions  that  are  identical  in  terms  of  spatial  prop¬ 
erties  (e.g.  space  between  humans  and  walls, 
or  between  obstacles  and  walls)  but  differ  im¬ 
portantly  in  social  implications  (e.g.,  passing 
between  a  person  and  a  store  window  to  which 
her  back  is  turned,  versus  passing  between  two 
characters  facing  each  other  at  a  conversa¬ 
tional  distance  vs.  passing  between  two  adver¬ 
tising  signs).  Choice  of  path  and  distance  be¬ 
tween  the  operator’s  avatar  and  obstacles 
(other  humans,  inanimate  objects  and  walls) 
are  interpreted  as  relating  to  participants’  ex¬ 
perience  of  the  ecological  validity  of  this  envi¬ 
ronment.  To  the  extent  users’  verbal  descrip¬ 
tions,  comments  and  navigational  decisions 
suggest  adequate  ecological  validity,  the  envi¬ 
ronment  is  considered  to  have  potential  as  a 
tool  for  use  in  assessing  and  intervening  with 
individuals  with  social  impairments,  including 
autism. 

The  controller  used  was  a  joystick  with  minimal 
displacement,  that  moved  the  avatar  in  the  di¬ 
rection  that  force  was  applied.  Participants  were 
trained  in  the  operation  of  the  controls  by  fol- 
lowing_a  path  through  large  virtual  mall  corri¬ 
dors,  during  which  they  were  instructed  to  avoid 
collision  with  walls  and  obstacles.  Next  they 
were  familiarized  with  the  layout  of  the  mall  by 
following  experimenter  instructions  to  navigate 
to  each  of  the  individual  stores.  The  target  loca¬ 
tion  task  entailed  navigating  to  four  different 
stores  to  locate  in  each  a  box  containing  a  col¬ 
ored,  numbered  sphere.  The  experimenter 
named  the  store  in  which  the  next  target  could 
be  found,  and  the  participant’s  task  was  to  lo¬ 
cate  the  box,  which  opened  once  s/he  was 
within  range,  and  to  read  out  the  number  and 
name  the  color.  Participants  performed  two 
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runs  of  the  target  location  task,  each  with  a  dif¬ 
ferent  order  of  shops,  and  runs  were  counter¬ 
balanced  across  participants. 

Participants  were  asked  to  ‘think  out-loud’  as 
they  performed  the  tasks.  This  instruction  was 
later  modified  to  “  Describe  what  you  are  do¬ 
ing.”  At  the  end  of  the  session,  participants 
were  invited  to  comment  on  the  experience  and 
identify  anything  they  liked  or  disliked. 

Individual  participant  comments  were  recorded 
and  were  coded  for  indication  of  perceived  real¬ 


ism  in  the  environment.  For  example,  use  of  the 
first  person  to  refer  to  one’s  own  avatar,  and 
imputation  of  agency  and/or  mental  state  to 
other  artificial  humans  were  treated  as  indica¬ 
tions  of  perceived  ecological  validity. 

Participants’  paths  through  the  environment 
were  recorded.  After  the  first  four  participants’ 
experience,  speech  output  was  added,  and  the 
distance  of  two  sets  of  obstacles  from  the  store 
entrances  they  blocked  was  increased.  'Visuali¬ 
zations  of  the  paths  taken  by  participants  before 
and  after  these  changes  were  implemented  are 


Figure  1 . 
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shown  in  Figure  1.  All  four  participants  using 
Version  1  (no  speech)  walked  between  the  two 
characters  who  were  facing  each  other,  both  on 
entering  and  exiting  the  store,  and.,  similarly, 
walked  between  the  two  inanimate  objects.  In 
contrast,  participants  who  used  the  Version  2  of 
the  mall,  to  which  speech  had  been  added, 
were  more  likely  to  walk  around  the  now  audibly 
conversing  characters,  rather  than  between 
them,  while  all  navigated  between  the  inani¬ 
mate  objects,  despite  the  increased  ease  of 
circumnavigating  both  pairs  of  obstacles. 

Figure  1:  Paths  taken  in  the  vicinity  of  social 
(virtual  humans)  vs.  non-social  (objects)  obsta¬ 
cles,  by  participants  experiencing  Versions  1 
and  2  of  the  Virtual  Mall.  Arrows  indicate  the 
direction  the  virtual  humans  are  facing.  In  Ver¬ 
sion  2  the  two  people  facing  each  other  can  be 
heard  to  converse.  In  addition,  in  Version  2, 
both  the  pair  of  people  and  the  pair  of  objects 
have  been  moved  farther  from  the  door,  so  that 
it  is  easier  to  go  around  both  rather  than  pass¬ 
ing  between  the  two  members  of  the  pair. 

Participants  were  able  to  learn  rapidly  to  navi¬ 
gate  the  mall,  and  locate  the  targets.  Review  of 
participants’  paths  and  their  verbal  comments 
during  and  after  participation  suggest  that  their 
navigational  decisions  reflect  their  awareness  of 
social  constraints.  They  used  the  first  person  in 
describing  their  avatar’s  movement  and  inten¬ 
tions,  irrespective  of  whether  they  experienced 
a  first-person  or  a  third-person  perspective. 
They  tended  to  ascribe  agency  and  even  men¬ 
tal  states  to  the  virtual  humans.  One  participant, 
passing  near  a  virtual  human,  said  “Excuse 
me!” 

In  addition,  they  tended  to  refrain  from  walking 
between  characters  who  were  facing  each 
other,  when  the  characters  were  audibly  con¬ 
versing.  The  gain  in  eliciting  socially  appropriate 
responses  from  addition  of  speech  is  a  promis¬ 
ing  indication  for  further  development  of  the 
mall  as  an  evaluative  and  training  tool.  Addi¬ 
tional  design  trials  are  planned  with  younger 
individuals  with  ASD  and  controls  matched  for 
nonverbal  mental  age. 


SIMULATION  TO  TRAIN  CONVERSATIONAL 
INTERACTION 

Lack  of  success  in  initiating  and,  especially,  in 
maintaining  a  conversation  is  a  hallmark  of 
ASD.  The  impoverished  opportunities  and  poor 
success  rate  of  peer  interaction  persists  into 
older  adolescence  and  adulthood,^^  presenting 
an  obstacle  to  the  casual  social  interaction  that 
is  a  part  of  much  of  human  activity.  Participating 
in  a  conversation  involves,  among  other  things, 
turn-taking,  gaining  awareness  of  the  topic  of 
the  conversation,  interpreting  the  partner’s  non¬ 
verbal  as  well  as  verbal  cues  and  knowing 
when  and  how  it  is  appropriate  to  change  the 
topic. 

The  Autism  Research  Group  is  collaborating 
with  SIMmersion  LLC™  to  develop  an  interac¬ 
tive  social  conversation  simulation  for  adults 
with  high-functioning  ASD. 

In  this  system,  the  simulated  character 
(represented  by  a  data-base  of  video  clips  per¬ 
formed  by  an  actor)  remembers  the  conversa¬ 
tion  history  and  responds  as  would  an  actual 
interlocutor  to  the  user’s  speech. 

Structure  is  provided  by  the  responses  offered 
for  the  user’s  choice.  The  simulated  character’s 
verbal  and  nonverbal  responses,  and  his  will¬ 
ingness  to  continue  the  interaction  provide  in¬ 
trinsic  feedback..  At  the  same  time,  cues  are 
available  from  a  help  agent  in  a  separate  win¬ 
dow  who  enthusiastically  applauds  when  the 
user  chooses  an  appropriate  question,  and  cov¬ 
ers  her  face  with  both  hands  when  the  user’s 
response  choice  is  inappropriate.  Each  play  of 
the  simulation  is  different,  the  interaction  is 
compelling,  and  there  is  the  possibility  of  im¬ 
proving  one’s  score  each  time.  These  game-like 
aspects  of  the  simulation  are  expected  to  help 
motivate  the  user  to  play  the  simulation  repeat¬ 
edly  and  thus  build  his/her  conversation  skills. 

The  goal  is  to  provide  the  individual  with  in¬ 
creasingly  successful  experience,  supported,  to 
the  extent  s/he  chooses,  with  simplified  instruc¬ 
tion  and  reviewing  of  parts  or  all  of  the  past 
conversation,  that  can  be  pursued  in  a  non¬ 
threatening  environment,  in  order  to  develop  a 
base  of  conversational  skills. 
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DISCUSSION 

Individuals  with  ASD  have  missed  out  on  much 
of  the  experience  required  for  social  and  cogni¬ 
tive  development,  possibly  because  of  disrup¬ 
tion  to  their  development  in  infancy  as  a  result 
of  heavy  genetic  liability  to  anxiety/depressive 
disorders  leading  to  very  early  onset  of  symp¬ 
toms  of  these  genetically  related  disorders 
(Trepagnier,  1996).  These  individuals  require 
training  appropriate  to  their  developmental  level 
to  improve  the  success  of  their  social  interac¬ 
tions,  thereby  raising  the  likelihood  that  they  will 
become  more  integrated  into  their  community. 
Computerized  and  virtual  reality  technologies 
enormously  broaden  the  scope  of  possible  in¬ 
terventions.  The  dual  challenge  is  to  make  tech¬ 
nology-based  interventions  both  efficacious  in 
inculcating  gains  in  social  skills  and  entertaining 
enough  to  overcome  autistic  motivational  barri¬ 
ers,  so  that  consumers  and  their  families,  edu¬ 
cators  and  clinicians  will  use  and  benefit  from 
them. 
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Abstract:  Virtual  reality  (VR)  exposure  therapy  has  been  shown  to  be  successful  in  treating  many 
types  of  specific  phobias  which  are  mostly  visual  in  nature.  However,  limited  research  has  been 
completed  on  the  use  of  VR  therapy  for  Blood  Injection-Injury  (Bll)  phobias,  one  of  the  subtypes  of 
Specific  Phobia  listed  in  the  DSM-IV  TR.  Since  Bll  phobia  may  operate  by  some  tactile  component, 
it  may  respond  differently  to  VR  therapy  compared  to  other  categories  of  specific  phobias  that  are 
largely  visually  activated.  This  paper  discusses  initial  development  and  results  from  a  study  on  both 
subjective  and  objective  arousal  elicited  by  a  prototype  virtual  world  which  has  been  developed  to 
treat  those  with  Bll  phobia.  The  present  study  evaluated  the  responses  of  20  healthy,  non-phobic 
male  and  female  participants  to  VR  blood  and  injection  stimuli.  Initial  results  are  positive  and  show 
that  the  VR  world  delivers  appropriate  cues  to  elicit  physiological  and  self-reported  arousal  when 
exposed  to  the  injection  scenarios.  Correlations  between  self-reported  anxiety  and  physiological 
arousal  confirm  that  individuals  experiencing  greater  symptoms  of  fear  in  conditions  involving  blood 
or  injections  will  exhibit  more  intense  arousal  from  the  virtual  stimuli  than  those  who  experience  re¬ 
duced  symptoms.  Findings  suggest  that  the  virtual  world  is  an  effective  method  of  cue  exposure  for 
individuals  who  experience  anxiety  in  situations  related  to  blood  and  injections.  Future  research  on 
the  use  of  VR  exposure  therapy  in  the  treatment  of  Bll  phobia  is  warranted. 


INTRODUCTION 

The  DSM-IV  TR  classifies  phobias  into  three 
groups;  1)  Agoraphobia,  2)  Social  Phobia,  and 

3)  Specific  Phobias.  The  further  subdivision 
within  Specific  Phobias  is:  1)  Blood-Injection- 
Injury  (Bll),  2)  Animal,  3)  Natural  Environment, 

4)  Situational,  and  5)  “Other”.  ^  Although  VR 
exposure  therapy  has  been  used  successfully 
for  more  than  a  decade  to  treat  phobias;  includ¬ 
ing  specific  phobias,  panic  disorder  and  agora¬ 
phobia,  and  social  phobia  only  one  study 
has  attempted  VR  usage  for  injection  phobia.^ 

Those  who  suffer  from  Bll  phobias  may  fear 
either  the  sight  or  the  pain  of  the  injection  and 
often  begin  avoidance  of  any  situation  which 
may  cause  exposure  to  an  injection,  such  as 
donating  blood  or  going  to  the  doctor.  In  ex¬ 
treme  cases,  the  phobic  may  even  begin  to  fear 
driving  past  a  hospital.^  Because  Bll  phobia 
involves  not  only  visual  stimuli  which  elicit  the 
phobic  response,  but  also  tactile  stimuli;  it  may 


differ  from  other  specific  phobias  in  how  it  will 
respond  to  VR.  In  the  previous  injection  study, 
it  was  found  that  VR  could  be  used  to  success¬ 
fully  elicit  subjective  (measured  with  Subjective 
Units  of  Distress  (SUDs  ratings)  and  physiologi¬ 
cal  arousal  (measured  with  heart  rate  (HR))  in 
those  with  needle  phobia,  however,  the  results 
indicated  that  the  addition  of  tactile  stimuli 
would  have  proven  to  be  more  advantageous. 
In  addition,  the  previous  study  presented  the 
VR  environment  via  a  desktop  display  instead 
of  a  more  immersive  head-mounted  display 
(HMD).  ^  The  present  study,  therefore,  sought 
to  investigate  elicitation  of  arousal  via  a  HMD 
VR  system  to  be  used  in  treating  those  with  Bll 
phobias.  As  an  initial  first  step,  however,  we 
have  sought  to  establish  a  baseline  with  those 
who  do  not  meet  the  DSM-IV  TR  criteria  for  a 
Bll  phobia  in  order  to  use  this  arousal  level  as  a 
baseline  comparison  for  phobic  participants 
exposed  to  the  VR  world. 


109 


Annual  Review  of  CyberTherapy  and  Telemedicine 


MATERIALS  AND  METHODS 

Participants 

Twenty  healthy,  non-phobic  participants  (13 
females  and  7  males)  from  the  San  Diego  area 
were  recruited  for  this  study.  Their  mean  age 
was  27  ±  10,  ranging  from  20  to  54  years  of 
age. 

Virtual  World 

The  low  cost  VR  system  was  developed  to  be 
used  on  a  personal  computer  (PC)  platform  with 
a  HMD  and  tracking  system  so  that  the  partici¬ 
pant  could  explore  the  world  via  head  tracking 
as  well  as  navigation  provided  via  a  joystick. 
The  virtual  world  was  modeled  after  an  outpa¬ 
tient  clinic  in  La  Jolla,  CA  and  is  almost  an  ex¬ 
act  replica  of  the  real  life  structure.  Participants 
can  navigate  through  the  clinic’s  entrance, 
lobby,  waiting  room  and  exam  rooms.  Exam 
rooms  are  filled  with  typical  phlebotomy  lab 
paraphernalia,  including  vials  of  blood.  In  one 
of  the  exam  rooms,  the  participant  may  sit  down 
and  have  a  “nurse  avatar”  administer  an  injec¬ 
tion.  This  is  viewed  in  first  person,  as  if  the  par¬ 
ticipant  is  seated  in  the  exam  room  chair  and 
viewing  his  arm  while  receiving  the  injection. 

Measurements 

Objective  physiological  measures  and  subjec¬ 
tive  self-report  data  were  collected  from  each 
participant.  Physiological  data  consisted  of 
heart  rate  and  multiple  respiration  measures 
collected  using  VivoMetric’s  wireless  Lifeshirt 
system;  and  heart  rate,  skin  conductance  and 
peripheral  skin  temperature  data  collected  us¬ 
ing  J  &  J  Engineering’s  C2  device.  Physiologi¬ 
cal  data  was  collected  at  baseline  and  continu¬ 
ously  in  real-time  throughout  exposure  to  the 
virtual  world.  Self-report  data  consisted  of  a 
Fear  Questionnaire,  a  Blood  Injection  Symp¬ 
toms  Scale  (BISS),  a  State  Blood  Injection 
Symptoms  Scale  (S-BISS),  and  a  Post- 
Experience  Questionnaire.  The  Fear  Question¬ 
naire  and  BISS  were  administered  before  base¬ 
line,  the  S-BISS  was  administered  pre-  and 
post-VR  exposure,  and  the  Post-Experience 
Questionnaire,  which  included  questions  on 
presence  and  simulator  sickness,  was  adminis¬ 
tered  after  VR  exposure. 


Design 

Ten  (10)  participants  were  randomly  assigned, 
using  a  random  numbers  table,  to  Group  A  and 
10  were  randomly  assigned  to  Group  B.  Group 
A  was  verbally  instructed  to  navigate  through 
the  virtual  clinic  to  a  specified  exam  room 
where  they  would  be  seated  but  was  not  in¬ 
formed  that  they  would  be  “receiving  an  injec¬ 
tion”.  Group  B  was  given  the  same  instructions, 
but  was  also  notified  that  they  would  receive  an 
injection.  Outcome  measures  were  analyzed 
between  groups  to  determine  effect  of  instruc¬ 
tions  on  anxiety  (e.g.  anticipatory  anxiety). 

All  20  participants  provided  consent  before 
completing  the  pre-test  questionnaires.  All  par¬ 
ticipants  completed  a  3  minute  baseline  during 
which  physiological  measures  were  continu¬ 
ously  collected.  After  baseline,  participants 
completed  the  S-BISS  and  each  group  (A  and 
B)  received  their  respective  instructions.  Par¬ 
ticipants  navigated  the  virtual  world  for  3  min¬ 
utes  and  physiological  data  was  collected 
throughout.  During  VR  exposure,  investigators 
recorded  time  markers,  indicating  when  the  in¬ 
jection  was  received  by  each  participant.  After 
VR  exposure,  participants  completed  the  post¬ 
test  questionnaires  and  subjective  feedback 
was  collected  by  investigators. 

RESULTS 

Physiological  data  from  both  conditions  (A  and 
B)  was  averaged  and  differences  between 
baseline  and  VR  were  compared  using  a  paired 
t-test  (a  =  0.05).  Analyses  indicated  a  signifi¬ 
cant  increase  in  skin  conductance  (df  =  1,  17;  p 
=  0.018)  and  respiration  rate  (df  =  1,  18;  p  = 
0.0003)  during  VR.  Temperature  also  in¬ 
creased  during  VR,  but  the  difference  was  not 
statistically  significant  (df  =  1,  18;  p  =  0.089). 
Combined  group  data  exhibited  a  decrease  in 
HRV  (df  =  1,  18;  p  =  0.057)  and  LF  (df  =  1,  18; 
p  =  0.099)  during  VR,  but  neither  difference  was 
statistically  significant.  T-tests  determined  that 
differences  in  heart  rate  (df=1,  18;  p=0.631), 
VLF  (df  =  1,  18;  p  =  0.672)  and  HF  (df  =  1,  18;  p 
=  0.0442)  measurements  were  not  significant. 

Combined  group  data  from  VR  exposure  was 
averaged  into  three  distinct  segments  and  ana¬ 
lyzed;  period  before  injection  (VR1),  injection 
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(VR2),  and  period  after  injection  (VR3).  A  re¬ 
peated  measures  ANOVA  analysis  of  baseline 
and  each  VR  segment  revealed  a  significant 
increase  in  skin  conductance  from  baseline  dur¬ 
ing  VR1  and  VR2.  Skin  conductance  also  in¬ 
creased  during  VR3,  but  the  change  from  base¬ 
line  was  not  significant.  A  similar  analysis  of 
respiration  rate  indicated  that  BPM  (breaths  per 
minute)  during  baseline  was  significantly  lower 
than  VR1  and  VR3,  but  not  significantly  differ¬ 
ent  from  VR2  (df  =  3,  51;  p<0.0001).  An  analy¬ 
sis  of  VLF  data  revealed  that  VR1  was  signifi¬ 
cantly  higher  than  baseline,  VR2,  and  VR3  (df  = 
3,  51;  p<0.0001).  Both  HF  (df  =  3,  51; 
p<0.0001)  and  LF  (df  =  3,  51;  p  =  0.0002)  data 
exhibited  a  statistically  significant  increase  dur- 
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ing  VR1  compared  to  VR2  and  VR3.  HRV  was 
lower  during  each  of  the  VR  segments,  indicat¬ 
ing  increased  anxiety,  but  differences  were  not 
significant.  Analysis  of  heart  rate  data  indicated 
no  significant  difference  between  baseline  and 
VR  segments  (df  =  3,  51;  p  =  0.256  and  df  =  3, 
51). 

Total  scores  from  the  BISS  were  positively  cor¬ 
related  to  a  change  in  skin  conductance  be¬ 
tween  baseline  and  VR  exposure.  Higher  self- 
reported  BISS  totals  were  correlated  to  a 
greater  change  in  skin  conductance  values  be¬ 
tween  baseline  and  VR  (r  =  0.541 1 ,  p=0.025). 
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Repeated  measures  ANOVAs  (a=0.05)  were 
used  to  evaluate  treatment  effects.  Analyses 
were  conducted  comparing  baseline  and  VR  as 
well  as  baseline,  VR1,  VR2,  and  VR3.  Analysis 
of  VLF  between  baseline  and  VR  revealed  sig¬ 
nificant  interaction  effects  between  conditions. 
HF  data  also  demonstrated  interaction  effects, 
but  the  interaction  was  not  found  to  be  signifi¬ 
cant.  An  evaluation  of  baseline  and  VR  seg¬ 
ment  averages  revealed  that  Group  B  exhibited 
a  larger  increase  in  skin  conductance  values 
from  baseline  than  Group  A,  but  the  difference 


was  not  significant.  All  other  measurements 
indicated  no  significant  interaction. 

DISCUSSION 

This  study  shows  a  general  pattern  of  arousal 
during  VR  exposure.  Skin  conductance  and 
respiration  rate  values  increased  from  baseline 
to  VR,  indicating  an  increase  in  arousal  due  to 
VR  exposure.  The  significant  difference  in  skin 
conductance  values  of  VR1  and  VR2  (but  not 
VR3)  compared  to  baseline  may  suggest  that 
the  initial  experience  of  the  virtual  world  and  the 
injection  were  the  most  stimulating  segments  of 
the  VR  exposure.  This  finding  may  also  indi¬ 
cate  that  participants  became  relaxed  after  the 
injection,  which  resulted  in  a  physiological  re¬ 
sponse  that  was  closer  to  baseline  levels. 

The  significant  change  in  skin  conductance  val¬ 
ues  during  VR1  and  VR2  may  also  be  evidence 
of  anticipatory  anxiety  exhibited  by  participants. 
Participants  may  have  been  stressed  by  the 
thought  of  visiting  a  hospital  clinic  and  the 
thought  of  an  injection  (Group  B  only),  which 
resulted  in  an  initial  increase  in  arousal  (VR1 
and  VR2)  and  a  subsequent  stabilization  of 
physiological  response  over  time  (VR3).  This 
interpretation  is  supported  by  the  differences 
exhibited  between  the  two  experimental  condi¬ 
tions.  While  no  statistically  significant  differ¬ 
ences  were  determined.  Group  B  revealed  a 
larger  increase  in  initial  skin  conductance  val¬ 
ues  from  baseline  compared  to  Group  A,  which 
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may  be  due  to  greater  anticipatory  anxiety  re¬ 
sulting  from  being  told  they  would  receive  an 
injection.  Group  A  was  not  notified  of  the  injec¬ 
tion  and  demonstrated  less  physiological 
arousal  during  initial  exposure  to  the  virtual 
world. 

The  respiration  rate  data  demonstrates  a  differ¬ 
ent  arousal  pattern  from  the  skin  conductance 
results,  but  provides  evidence  of  general 
arousal  during  VR  exposure  that  is  similar  to 
the  skin  conductance  data.  The  significant  in¬ 
crease  in  BPM  during  VR1  and  VR3  compared 
to  baseline  suggests  that  the  virtual  world  pro¬ 
duced  a  sustained  state  of  heightened  arousal 
in  study  participants.  Unlike  skin  conductance 
data,  BPM  measures  suggest  that  participant’s 
physiology  did  not  return  to  baseline  levels  after 
the  injection,  as  BPM  remained  significantly 
higher  than  baseline  at  the  beginning  and  end 
of  VR  exposure.  The  non-significant  increase 
in  BPM  during  VR2  can  be  explained  by  many 
factors  and  may  provide  evidence  that  partici¬ 
pants  held  their  breath  while  the  shot  was  ad¬ 
ministered.  Generally,  lower  BPM  correlates  to 
reduced  arousal,  but  this  association  is  incon¬ 
sistent  with  the  rest  of  the  collected  data,  as  VR 
exposure  produced  heightened  arousal.  Con¬ 
sequently,  the  lower  respiration  rate  during  the 
injection  (VR2)  may  be  better  interpreted  by 
behavioral  factors,  like  holding  one’s  breath, 
rather  than  state  of  arousal. 


Heart  rate  increases  were  not  found  to  be  sta¬ 
tistically  significant  between  baseline  and  VR 
exposure.  This  has  also  been  found  in  other 
VR  studies  which  concluded  that  VR  exposure 
may  not  be  powerful  enough  to  effectively  trig¬ 
ger  the  behavioral  activation  system  (BAS), 
measured  here  via  heart  rate,  but  does  suc¬ 
cessfully  trigger  the  behavioral  inhibition  system 
(BIS)  which  is  measured  here  via  skin  conduc¬ 
tance.  There  was,  however,  some  significance 
found  when  looking  at  different  frequency 
bands  of  heart  rate  variability,  indicating  that 
this  measure  may  prove  sensitive  enough  for 
future  studies. 

This  study  also  shows  a  significant  correlation 
between  self-reported  anxiety  and  objective 
physiological  measures  of  anxiety.  Participants 
who  scored  higher  on  the  BISS  indicated  that 
they  were  generally  more  anxious  about  blood 
and  injection  stimuli  (though  not  reaching  levels 
diagnosable  as  phobic),  which  was  consistent 
with  the  increase  in  skin  conductance  levels 
during  VR  exposure.  This  association  suggests 
that  participants  who  report  greater  symptoms 
of  fear  in  situations  involving  blood  or  injections 
will  exhibit  a  larger  physiological  response  to 
the  virtual  world  than  those  reporting  reduced 
symptoms.  This  finding  also  reveals  the  effi¬ 
cacy  of  the  virtual  environment  at  providing  po¬ 
tent  blood  and  injection  cues,  and  at  generating 
an  appropriate  psychological  and  physiological 
stimulus  response  in  participants  who  report 
symptoms  of  fear. 

CONCLUSION 

In  this  study  we  determined  that  the  virtual  cues 
produced  significant  changes  in  physiological 
arousal  in  normal  non-phobic  participants.  In¬ 
creased  arousal  levels  resulting  from  VR  expo¬ 
sure  suggest  that  the  virtual  world  is  an  effec¬ 
tive  method  of  cue  exposure  for  individuals  who 
report  symptoms  of  fear  in  situations  involving 
blood  or  injections.  However,  the  results  of  this 
study  are  not  conclusive  and  additional  analysis 
must  be  completed  to  better  determine  the 
cause  of  the  physiological  changes  that  result 
from  VR  blood  and  injection  stimuli.  In  addition, 
future  research  should  evaluate  the  capacity  to 
which  the  virtual  environment  can  be  general¬ 
ized  to  larger  populations,  including  phobic  indi¬ 
viduals. 
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FUTURE  WORK 

Future  studies  should  include  the  presentation 
of  the  current  VR  world  to  those  who  meet 
DSM-IV  TR  criteria  for  Bll  phobia.  In  addition, 
comparison  studies  should  include  presentation 
via  both  a  flat  screen  and  a  HMD  to  determine  if 
indeed  the  more  immersive  HMD  increases 
presence,  immersion,  and  arousal.  A  compari¬ 
son  study  should  also  be  included  to  determine 
if  tactile  feedback  done  in  reality  which  corre¬ 
sponds  to  the  VR  visual  stimuli  would  increase 
initial  arousal  and  possibly  result  in  a  more  effi¬ 
cient  method  of  treatment.  Finally,  given  ad¬ 
vances  in  wireless  technology  for  physiological 
monitoring,  it  would  be  interesting  to  test  partici¬ 
pants  in  both  the  actual  real  world  setting  and 
compare  results  to  those  elicited  in  VR  expo¬ 
sure. 
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Working  Things  Out: 

A  Therapeutic  Interactive  CD-Rom  Containing  the  Stories  of  Young 
People  Dealing  with  Depression  and  other  Mental  Health  Problems 

E.  Brosnan,  Dr.  J.  Sharry,  Prof.  C.  Fitzpatrick 

University  College  Dublin/  Mater  Hospital 


Abstract;  It  is  widely  accepted  that  children  and  young  people  enjoy  using  computers.  Seymour  Pa- 
pert^  suggests  that  'it  is  timely  for  school  counsellors  and  child  therapists  to  bridge  the  digital  genera¬ 
tion  gap  and  innovate  with  computers  in  their  work  with  children  of  the  digital  age.  ’  Working  Things 
Out  (WTO)  was  developed  in  response  to  the  growing  need  for  computer-based  resources  for  ado¬ 
lescents  in  therapy.  It  is  an  interactive  CD  ROM/DVD  developed  as  a  means  of  engaging  adoles¬ 
cents  about  mental  health  issues  by  giving  them  information  in  the  form  of  animated  personal  stories 
told  by  other  young  people  dealing  with  problems  such  as  depression,  bullying,  eating  problems  and 
self-harm.  The  CD-ROM/DVD  is  accompanied  by  a  manual  for  professionals  on  how  to  use  the  re¬ 
source. 

With  the  help  of  therapists,  and  in  collaboration  with  graphic  designers,  animators  and  multimedia 
professionals,  the  1 1  young  people  who  participated  in  the  project  told  their  stories,  narrated  in  their 
own  voice  and  illustrated  by  graphics  and  animation.  ‘Working  Things  Out’  is  currently  being  used  as 
an  educational  and  therapeutic  tool  with  other  adolescents  at  risk  of  mental  health  problems,  both  as 
a  way  of  engaging  young  people  to  reflect  about  mental  health  issues  and  as  a  means  of  inviting 
them  to  tell  their  own  story.  The  paper  describes  the  background  and  development  of  the  'Working 
Things  Out'  project,  including  samples  from  the  stories  and  a  description  of  how  the  CD-ROM/DVD 
can  be  used  in  psychotherapy. 
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THERAPEUTIC  STORY  TELLING  AND  DIGI¬ 
TAL  MEDIA 

Telling  and  listening  to  stories  is  a  fundamental 
human  activity,  both  as  a  means  of  entertain¬ 
ment  and  communicating  information.  Whether 
we  are  listening  to  a  personal  anecdote,  reading 
a  novel  or  watching  a  soap  opera,  a  ‘good  story’ 
holds  much  greater  appeal  than  a  series  of  facts 
alone  and  is  much  more  likely  to  engage  our 
imagination  and  hold  our  attention.  Personal 
stories  that  relate  real-life  accounts  of  peoples 
lives,  and  which  have  the  mark  of  authenticity 
can  hold  a  particular  appeal  to  an  audience, 
allowing  special  identification  with  the  character 
and  context.  Further,  personal  stories  can  have 


great  meaning  to  the  storytellers  as  well  as  the 
listeners  in  that  the  act  of  telling  the  story  can 
shape  and  define  their  lives. 

As  a  result  psychotherapists  are  very  interested 
in  personal  stories.  Psychotherapy  can  be  con¬ 
ceived  as  process  of  inviting  clients  to  tell  and 
retell  their  life  story  from  a  variety  of  perspec¬ 
tives  with  the  aim  of  reaching  a  coherent  and 
meaningful  narrative  at  the  end.^  Within  a 
strengths-based  approach  to  psychotherapy  the 
process  can  be  conceived  as  helping  clients 
shift  from  initially  self-limiting  and  problem- 
focused  accounts  of  their  lives  to  more  positive 
and  strengths-oriented  accounts  that  are  more 
liberating  and  empowering.^’'  For  example,  a 
person  may  begin  psychotherapy  by  telling  the 
story  of  how  he  became  depressed  and  how 
this  damages  his  life  and  end  therapy  with  a 
‘new’  story  of  how  he  has  coped  with  the  de¬ 
pression  and  how  this  leads  to  new  possibilities 
in  his  life.  This  is  not  simply  a  semantic  differ- 
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ence:  the  act  of  retelling  your  story  to  an  em- 
pathic  listener,  in  a  way  that  identifies  new 
strengths-based  ‘plot  lines'  and  that  incorpo¬ 
rates  supportive  ‘characters’  more  centrally  to 
the  story,  is  in  itself  beneficial  and  transforming 
to  ones  personal  identity. 

Computers  and  digital  media 

Computers  and  digitai  media  provide  therapists 
with  new  means  to  engage  clients  and  assist 
them  in  telling  their  life  stories.  Narrative  struc¬ 
tures  and  storylines  are  widely  employed  in 
many  different  types  of  digital  media,  including 
video  game  design,  web-design,  and  e- 
learning.  By  including  a  narrative  structure,  this 
can  increase  the  appeal  for  users  and  offer  a 
context  for  creating  meaning,  thereby  deepen¬ 
ing  their  experiences  of  the  information  being 
presented.  Abbe  Don  in  an  article  entitled 
“Narrative  and  the  Interface”  argues  that  com¬ 
puters  can  play  in  modern  society  the  role  of 
the  storyteller  of  oral  cultures.®®  The  computer 
becomes  the  processor  of  the  “told”  information, 
the  user/viewer  takes  on  the  information  and 
interprets  it  in  a  way  that  is  meaningful  to  them. 

Digital  storytelling  techniques  are  particularly 
relevant  in  the  context  of  psychotherapy.  The 
computer  provides  novel  means  of  both  ex- 
pressing  and  iistening  to  therapeutic  stories. 
The  computer  facilitates  the  expression  of  the 
therapeutic  story  by  ailowing  the  incorporation 
of  sound,  image,  animation  as  well  as  text  and 
verbal  communication  in  the  teliing.  Equally  the 
computer  facilitates  the  iistening  to  the  story,  by 
providing  interactive  interfaces  and  databases 
that  aliow  the  listener  to  pace  the  story  to  their 
own  need  and  to  make  choices  about  what  is 
relevant.  The  Working  Things  Out  project  aimed 
to  build  on  these  twin  key  potentials  that  are 
afforded  by  digital  media  in  therapeutic  storytel¬ 
ling. 

WTO  PROJECT  RATIONALE  AND  BACK¬ 
GROUND 

The  idea  for  the  WTO  project  emerged  during 
the  ‘Challenging  Times’  research  study®  which 
looked  at  the  incidence  of  depression  and  other 
mental  health  difficulties  amongst  the12-15  year 
old  population  in  secondary  schools  in  the 
North  Dublin  City  area.  A  subsequent  qualita¬ 


tive  study®  engaged  a  smaller  group  of  young 
people  to  share  their  stories  of  how  they  were 
coping  and  dealing  with  depression  and  other 
difficult  life  experiences.  A  key  finding  of  the 
studies  was  the  fact  many  young  people  in  the 
community  who  suffered  from  depression  did 
not  access  traditional  professional  mental 
health  services. 

The  WTO  project  grew  out  of  a  desire  to  create 
an  accessible  means  of  providing  mental  health 
information  as  well  as  a  means  of  engaging 
young  people  in  professional  services  and  psy¬ 
chotherapy.  To  ensure  this  was  relevant  to 
young  people,  participants  from  the  study  were 
invited  to  contribute  to  the  making  of  the  CD- 
ROM/DVD  and  to  share  their  personal  stories  of 
dealing  with  a  difficult  life  experience  during 
adolescence.  The  inclusion  of  real  young  peo¬ 
ple’s  experiences  in  a  story  format  makes  the 
material  much  more  real  and  relevant.  A  multi- 
media  interactive  CD-Rom  was  chosen  as  the 
format  for  the  collection  of  stories  as  this  moti¬ 
vated  the  young  people  tell  their  story,  (many 
were  keen  to  gain  experience  of  creatively  us¬ 
ing  digital  media  creatively)  and  also  provided 
an  engaging  ‘youth  centred’  format  for  future 
users. 

The  project  was  developed  as  a  partnership 
between  the  Department  of  Child  and  family 
Psychiatry  in  the  Mater  Hospital,  the  charity 
Parents  Plus  (which  will  distribute  the  CD-Rom) 
and  the  Therapeutic  Technologies  Group  in 
Media  Lab  Europe. 

Development 

Young  people  aged  13  to  16  who  attended  the 
Mater  Child  and  Adolescent  Mental  Health  Ser¬ 
vice  and/  or  who  participated  in  the  Challenging 
Times  research  study  above  were  invited  to 
participate  in  the  project.  Specifically,  they  were 
asked  to  share  their  experience  and  stories  in 
order  to  make  a  CD-ROM  to  help  young  people 
overcome  problems  in  their  lives.  The  altruistic 
nature  of  the  project  appealed  to  many  of  them, 
as  much  as  the  possibility  of  working  with  multi- 
media  to  tell  their  stories.  After  a  series  of  gen¬ 
eral  focus  groups,  eleven  young  people  were 
selected  to  take  part.  The  problems  the  young 
people  were  dealing  with  ranged  from  more 
‘everyday’  issues  such  as  bullying,  conflict  with 
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parents  and  school  problems  to  more  specialist 
problems  such  as  self-harm,  bereavement,  ob¬ 
sessive-compulsive  disorder,  and  eating  prob¬ 
lems. 

To  develop  a  ‘script’  for  the  stories,  each  young 
person  attended  a  series  of  individual  meetings 
with  a  therapist  who  through  a  process  of  listen¬ 
ing  and  clarifying  helped  the  young  people  be¬ 
gin  to  tell  their  story  in  a  structured  format.  This 
structure  focused  not  only  on  their  experience 
of  having  a  problem,  but  also  how  they  coped 
and  what  supports  and  resources  helped  them. 
The  final  scripts  for  the  short  movies  were  co¬ 
authored  by  therapist  and  young  person.  Some 
young  people  were  able  to  write  the  majority  of 
the  script  themselves  and  others  needed  more 
support  (e.g.  the  therapist  would  transcribe  the 
main  points  from  a  recorded  interview  and  re¬ 
read  to  the  young  person  for  approval). 

The  young  people  were  also  involved  in  the 
creative  aspects  of  the  production.  Some  were 
involved  in  the  storyboarding  of  the  scenes,  the 
development  of  the  graphics  and  the  choice  of 
images.  Alongside  the  individual  sessions,  sev¬ 
eral  workshops  were  conducted  with  the  young 
people  on  animation,  graphics,  photography 
and  digital  audio.  This  helped  each  young  per¬ 
son  take  ownership  of  his/her  story  both  in  its 
script  and  in  how  it  would  be  animated  and  rep¬ 
resented,  A  key  aspect  of  this  project  was  to 
ensure  that  the  stories  are  personal  and  mean¬ 
ingful  to  the  participating  adolescents  and  their 
families.  The  introduction  to  the  stories  was 
written  in  conjunction  with  the  young  people 
and  is  as  follows; 

Welcome  to  'Working  Things  Out'  the  stories  of 
1 1  young  people  who  have  gone  through  some 
difficult  life  experiences.  By  sharing  our  stories, 
our  feelings  and  our  ways  of  coping  with  our 
problems,  we  hope  that  others  who  might  be 
going  through  a  similar  hard  time  will  be  re¬ 
minded  that  they  are  not  alone  and  that  there 
are  many  different  things  you  can  do  to  make 
things  better. 

Telling  our  stories  definitely  made  us  feel  better, 
it  was  good  to  get  stuff  off  our  chests.  We  hope 
that  the  very  real  experiences  you  hear  will  be  a 
source  of  support  and  learning  for  you.  Our 
stories  are  ongoing  and  not  all  our  problems 


have  gone  away  but  the  way  we  understand 
and  deal  with  them  has  changed.  While  we 
don’t  have  all  the  answers,  we  might  have 
some  useful  messages  and  information. 

In  developing  the  project  we  were  keen  to  en¬ 
sure  that  the  final  accounts  were  confidential 
meaning  that  we  would  not  use  any  identifying 
material  such  as  pictures  or  real  names.  Thus, 
in  order  to  maintain  the  authenticity  of  the  pro¬ 
ject,  the  young  people  themselves  provided  the 
‘voice  over’  recording  to  a  script  that  had  been 
co-written  with  the  therapists.  The  final  produc¬ 
tion  includes  ten  mini-movies  of  the  young  peo¬ 
ples’  stories  and  one  recorded  song  written  by 
one  participant  who  preferred  to  share  his  ex¬ 
periences  in  lyrics  and  music. 

Case  Example  -  Michelle’s  Story 

To  illustrate  the  content  and  purpose  of  WTO 
we  reproduce  below  the  opening  text  for  Mi¬ 
chelle’s  story  who  had  been  referred  to  an  ado¬ 
lescent  mental  health  service  due  to  her  eating 
problems. 

‘Shocked  Into  Reality’  -  Other  kids  at  school 
were  calling  me  fat. 

Other  kids  at  school  were  calling  me  fat.  I  was 
only  ten  at  the  time.  I  started  feeling  fat  and 
paranoid  and  thought  everybody  was  looking  at 
me.  Like  if  I  went  shopping  with  the  other  girls 
in  my  class  and  one  of  them  said  to  another  “ 
oh  those  Jeans  would  fit  you”  I  would  think  it 
was  directed  at  me  like,  that  she  meant  they 
wouldn’t  fit  me.  I  used  to  think  that  comments 
like  that  were  aimed  at  me,  though  they  would¬ 
n’t  be  in  reality.  I  started  skipping  breakfast 
purposefully  and  would  try  not  to  eat  much  at 
lunch.  Like  I’d  eat  half  a  cracker  at  break  and 
the  other  half  at  lunch.  I  used  to  think  that  if  I 
ate  any  more  Td  get  fat.  Then  I  started  exercis¬ 
ing,  doing  sit  ups  and  making  a  big  effort  in  PE 
so  Td  be  thinner. 

This  was  going  on  for  a  few  months  and  my 
Mam  noticed  I  was  losing  weight. 

She  took  me  to  the  doctor.  The  doctor  talked  to 
me  about  my  food  intake  and  told  me  I  should 
be  eating  more.  My  Mam  and  my  sister  were 
telling  me  the  same  thing  but  I  didn’t  listen  to 
any  of  them.  I  stiil  wanted  to  be  thinner.  I  didn’t 
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think  I  was  losing  weight  myself.  When  I’d  look 
in  the  mirror  I  used  to  see  myseif  as  bigger  than 
I  actually  was.  My  friends  started  saying  I  was 
too  skinny  but  I  didn’t  believe  them  either.  I 
wouid  feel  the  hunger  in  my  stomach  but  I  ig¬ 
nored  it  because  I  was  determined  not  to  eat 
and  I’d  feel  the  cold  even  when  it  was  sunny 
out 

Michelle’s  story  is  played  as  a  movie  and  illus¬ 
trated  with  line  graphics  that  support  her  narra¬ 
tion  (fig  1).  In  later  parts  of  the  story  she  goes 
on  to  say  how  she  and  her  mother  finally  sought 
help  for  her  difficulties.  A  turning  point  comes 
when  in  her  own  words  she  is  'shocked  into 
reality’  by  a  medical  report  which  highlights  just 
how  unden/veight  and  undernourished  she  is.  At 
the  end  of  the  story  she  relays  how  she  cur¬ 
rently  maintains  her  eating  and  health. 

USING  WTO  AS  AN  EDUCATIONAL  GUIDE 
WITH  YOUNG  PEOPLE 

As  well  as  being  beneficial  to  the  participants, 
the  WTO  CD  ROM  has  been  designed  as  an 
educational  guide  for  young  people  (and  their 
families)  who  are  experiencing  similar  prob¬ 
lems.  The  current  version  of  “Working  Things 
Out”  is  designed  to  be  used  in  a  facilitated  way 
by  professionals  working  with  young  people 
either  individually  or  with  a  small  group  of  ado¬ 
lescents,  in  either  a  mental  health  or  school 
setting.  The  aim  is  that  on  viewing  the  stories 
adolescents  may  feel  a  resonance  or  empathy 


with  the  story  teller  helping  them  feel  ‘not  alone’ 
if  similar  issues  affect  them.  In  addition,  viewing 
the  story  could  provide  an  important  focal  point 
for  discussion  and  also  engage  the  adolescent 
to  open  up  and  share  their  own  experience  if 
that  is  appropriate. 

The  final  version  of  the  CD-ROM  contains  an 
introduction  and  a  panoramic  interface  (fig  2), 
representing  a  street,  which  they  can  scroll 
across  left  and  right  to  view  the  silhouette  of 
eleven  young  people  whose  pseudonyms  are 
given  along  with  a  title  of  their  story.  Selecting  a 
story  allows  it  to  be  played  in  a  linear  fashion 
from  beginning  to  the  end,  with  the  options  of 
pause,  fast  fon/vard  and  rewind.  The  stories  are 
illustrated  by  graphics  and  animation  and  nar¬ 
rated  by  the  personal  voice  of  the  young  per¬ 
son.  In  addition,  each  story  is  punctuated  by 
several  ‘information  points’  which  provide  a 
commentary  on  some  of  the  key  points  raised  in 
the  story  as  well  as  some  sample  questions  for 
professionals  to  use  to  promote  reflection  and 
to  facilitate  the  personalisation  of  the  material. 
For  example,  the  opening  information  point  for 
Michelle’s  story  above  is  as  follows 

Pressure  to  be  thin 

Like  Michelle,  many  young  people  feel  self- 
conscious  about  their  appearance  and  body 
shape  and  feel  under  pressure  to  look  a  certain 
way  and  to  be  a  certain  weight.  For  Micheile  it 
became  a  probiem  and  she  began  to  see  her¬ 
self  as  ‘bigger  than  she  actually  was’,  and  she 
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Figure  2.  Panoramic  Opening  interface 


started  controlling  her  food  intake  to  the  point 
where  she  ignored  the  hunger  and  cold.  Like 
many  people,  Michelle  was  using  control  over 
food  to  cover  up  the  upset,  loneliness  and  sad¬ 
ness  she  was  feeling.  However,  when  a  young 
person  begins  to  ignore  the  reality  of  his/  her 
weight  loss  and  discomfort  and  get  obsessed 
about  limiting  food  intake  he/she  is  in  danger  of 
developing  a  serious  eating  disorder. 

Questions 

How  real  are  the  pressures  on  young  people 

to  diet  so  as  to  be  thin? 

If  you  were  a  friend  of  Michelle’s  what  could 

you  do  to  help  her? 

DiSCUSSiON  AND  EVALUATION 

Following  completion  of  the  Working  Things  Out 
Project  the  young  people  who  participated  in  it 
took  part  in  an  evaluation  of  the  process.  They 
all  reported  it  as  a  beneficial  experience.  Aside 
from  it  being  therapeutic  for  them  to  tell  their 
story  in  a  creative  form,  it  has  also  provided 
them  with  the  opportunity  to  learn  new  skills 
and  to  use  multimedia.  For  many,  participation 
has  also  had  positive  impacts  on  their  families. 
When  parents  have  reviewed  their  children’s 
story  it  has  often  opened  up  family  communica¬ 
tion.  One  mother  described  how  proud  she  was 
of  what  her  daughter  had  achieved  and  review¬ 
ing  the  story  became  a  special  moment  of  con¬ 
nection  for  them  both.  Finally,  participation  in 
the  CD  Rom  has  copper-fastened,  for  many  of 
the  young  people,  their  coping  skills  and  sur¬ 
vival  strategies.  They  have  moved  from  being 
‘recipients’  of  mental  health  information  to  being 
‘teachers’  and  those  who  provide  it. 

The  Working  Things  Out  CD-Rom/DVD  and 
Resource  Manual  was  launched  in  February 
2005.  The  package  was  field  tested  with  young 
people  attending  the  Mater  Hospital  both  indi¬ 
vidually  and  in  small  groups. 


Professionals  who  have  used  the  stories  with 
young  people  have  found  it  a  useful  engage¬ 
ment  tool  with  young  people,  inviting  them  to 
begin  to  talk  about  their  own  experience,  par¬ 
ticularly  when  the  viewed  story  resonates  with 
the  particular  experience  of  the  adolescent.  Ini¬ 
tial  cautions  are  to  ensure  that  the  story  shown 
is  relevant  to  the  young  persons  concerns  and 
context. 

It  is  our  experience  that  computers  and  multi- 
media  can  positively  engage  young  people  in 
coming  to  therapy.  There  is  anecdotal  evidence 
to  suggest  that  the  initial  ‘resistance’  to  coming 
to  therapy  often  expressed  by  adolescents  dis¬ 
sipates  when  the  computer  is  used  as  an  en¬ 
gagement  tool. 

FUTURE  DEVELOPMENTS 

A  further  project  has  now  developed  out  of  the 
Working  Things  Out  research.  As  part  of  the 
Therapeutic  Technologies  Research  Group 
(University  College  Dublin/Mater  Hospital)  we 
are  current  developing  “Transforming  Stories” 
which  is  a  database  driven  multimedia  storytel¬ 
ling  tool  delivered  both  online  and  on  stand 
alone  platform.  The  software  currently  being 
developed  will  allow  the  user  to  choose  a  story 
template  and  build  up  the  story  plot  using  cus¬ 
tomisable  characters  and  background  scenes 
adding  in  their  own  voiceover  and  animation. 
The  user  has  a  drag  and  drop  facility  for  placing 
story  objects  and  further  features  such  as  posi¬ 
tioning  tools,  playback  and  story  editing.  Other 
options  include  building  from  seed  stories 
where  they  can  change  the  plotline  and  add  in 
their  own  characters  and  dialogue.  In  the 
method,  the  therapist  introduces  the  idea  of 
making  a  story  and  shows  examples  of  what  is 
possible  using  the  software.  It  can  be  a  story 
tailored  to  a  problem  that  needs  resolution  or  a 
general  story  that  brings  about  some  positive 
change  in  the  storyline.  A  key  therapeutic  fea¬ 
ture  is  a  notebook  facility  where  the  child  and 
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therapist  keep  a  log  of  their  thoughts  on  the 
process  and  their  analysis  of  the  story.  This 
allows  for  reflection  and  gives  the  therapist  the 
opportunity  to  prompt  new  ideas  for  problem 
solving.  Finally  they  can  upload  and  share  their 
stories  to  an  online  community.  Over  time  it  is 
hoped  that  a  large  database  of  stories  will  be 
build  up  as  a  shared  resource  that  can  be  made 
available  to  young  people  struggling  with  prob¬ 
lems  as  a  source  of  support  and  reliable  infor¬ 
mation. 


Contact 
Dr  John  Sharry 

Dept,  of  child  and  family  psychiatry 
Mater  Hospital 
Dublin  7  Ireland. 

Tel;  +353  1  803  4793 

Eileen, brosnan@tcd.ie 
www.workinqthinqsout.ie 
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Assessment  of  Emotional  Reactivity  Produced  by  Exposure  to  Virtual 
Environments  in  Patients  with  Eating  Disorder 

M.  Ferrer-Garcia  &  J.  Gutierrez  Maldonado,  PhD. 

University  of  Barcelona 


Abstract:  This  paper  describes  the  effectiveness  of  virtual  environments  to  elicit  emotional  re¬ 
sponses  in  eating  disordered  patients.  This  study  is  part  of  a  wider  research  project  analysing  the 
influence  of  the  situation  to  which  subjects  are  exposed  on  their  performance  on  body  image  estima¬ 
tion  tasks. 

Though  it  seems  self-evident  that  there  is  a  close  relation  between  eating  disorders  (ED)  and  altered 
body  Image,  in  the  literature  on  the  subject  no  clear  association  has  in  fact  been  established,  and  in¬ 
deed  the  results  of  the  studies  are  often  contradictory  or  inconclusive.  A  number  of  hypotheses  have 
been  proposed  to  account  for  these  results.  Some  authors  have  stressed  the  fact  that  body  image  may 
be  to  a  certain  extent  more  a  state  than  a  trait  and  may  change  according  to  situational  or  emotional 
variables.  Several  studies  have  analysed  the  possible  impact  of  exposure  to  specific  objects  or  situa¬ 
tions  on  the  stability  of  the  body  image. 

In  this  study  we  designed  several  virtual  environments  that  were  emotionally  significant  for  subjects 
with  ED  in  order  to  generate  different  levels  of  anxiety  and  variations  in  mood.  Unlike  conventional 
methods  (real  exposure  to  the  situation,  exposure  to  photographs,  exposure  via  guided  imagination, 
and  so  on),  virtual  reality  exposes  subjects  to  interactive  three-dimensional  environments  that  simulate 
real  situation.  These  environments  have  ecological  validity  but  also  permit  strict  control  over  the  vari¬ 
ables  and  the  recording  of  data.  Virtual  reality  offers  many  of  the  advantages  of  the  conventional  meth¬ 
ods  mentioned  above,  and  also  overcomes  many  of  their  drawbacks. 

Thirty  female  patients  with  eating  disorders  were  exposed  to  six  virtual  environments:  a  living-room 
(neutral  situation),  a  kitchen  with  high-calorie  food,  a  kitchen  with  low-calorie  food,  a  restaurant  with 
high-calorie  food,  a  restaurant  with  low-calorie  food,  and  a  swimming-pool.  After  exposure  to  each 
environment  the  STAI-S  (a  measurement  of  state  anxiety)  and  the  CDB  (a  measurement  of  depres¬ 
sion)  were  administered  to  all  subjects. 

The  results  showed  significantly  higher  levels  of  state  anxiety  in  the  kitchen  with  high-calorie  food 
(F=1 3.120;  p  =  0.001),  the  restaurant  with  high-calorie  food  (F  =  14,  954;  p  =  0.001)  and  the  swim¬ 
ming-pool  (F  =  4.230;  p  =  0.049)  than  in  the  neutral  environment.  Analysing  the  scores  for  depres¬ 
sion  obtained  on  the  CDB,  significant  differences  again  appeared  between  the  high-calorie  food  en¬ 
vironments  (F  =  7.187;  p  =  0.012  in  the  kitchen  and  F  =  5.933;  p  =  0.021  in  the  restaurant)  and  the 
neutral  environment.  In  the  high-calorie  food  situations  patients  with  ED  showed  a  more  depressed 
mood. 

Virtual  reality  thus  appears  to  be  a  valid  instrument  particularly  useful  for  simulating  everyday  situa¬ 
tions  that  may  provoke  emotional  reactions  such  as  anxiety  and  depression,  in  patients  with  ED.  Vir¬ 
tual  environments  in  which  subjects  are  obliged  to  ingest  high-calorie  food  provoke  the  highest  levels 
of  state  anxiety  and  depression.  Previous  studies  have  shown  the  capacity  of  VR  to  elicit  states  of 
anxiety  in  patients  with  other  pathologies  too. 
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INTRODUCTION 

During  the  last  four  decades,  body  image  distur¬ 
bances  have  been  considered  one  of  the  main 
features  of  eating  disorders.  Nevertheless,  the 
published  literature  has  not  established  a  dear 
association  between  Eating  Disorders  (ED)  and 
altered  body  image,  and  found  results  have 
been  often  contradictory  or  inconclusive. 

In  order  to  explain  these  results  some  authors 
have  stressed  the  fact  that  body  image  may  be 
considered  more  a  state  than  a  trair^^^'^'*  and 
may  change  according  to  situational  or  emotional 
variables.  Several  studies  have  analysed  the 
possible  impact  of  exposure  to  specific  objects  or 
situations  on  the  stability  of  the  body  image. 
1,3,6,7,9,10  j^ggg  studies  suggest  that  body  image, 
or  some  of  its  components,  can  indeed  be  under¬ 
stood  as  a  state. 

Following  the  same  research  line,  we  want  to 
study  whether  exposure  to  virtual  environments 
produces  variations  in  the  estimation  of  body 
image  and  whether  anxiety  exerts  a  mediating 
role  in  this  relation.  With  this  aim,  we  designed 
several  virtual  environments  that  were  emotion¬ 
ally  significant  for  subjects  with  ED  in  order  to 
generate  different  levels  of  anxiety  and  varia¬ 
tions  in  mood.  This  paper  describes  the  first 
stage  of  the  project,  in  which  we  evaluate  the 
effectiveness  of  these  virtual  environments  to 
elicit  emotional  responses  in  ED  patients. 

In  the  area  of  eating  disorders,  virtual  reality 
has  been  used  for  the  assessment  and  treat¬ 
ment  of  body  image  disorders^ Perpina  and 
co-workers  (1999)  compared  the  efficacy  of  a 
traditional  body  image  disorders  treatment  pro¬ 
gram  with  the  efficacy  of  a  treatment  program 
that  included  virtual  exposure.  Patients  treated 
with  virtual  reality  showed  a  significantly  major 
recovery  in  body  image  disorder  and  in  depres¬ 
sive  and  anxious  sintomathology. 

These  studies  show  virtual  reality  as  a  valuable 
technology  for  psychopathological  assessment 
and  treatment.  One  of  the  main  advantages  of 
this  technology  is  its  capability  of  simulating  real 
situations  (high  ecological  validity)  and  allowing 
a  severe  control  of  the  variables,  at  the  same 
time.  So,  virtual  reality  offers  many  of  the  ad¬ 
vantages  of  the  conventional  methods  (real  ex¬ 
posure  to  the  situation,  exposure  to  photo¬ 


graphs,  exposure  via  guided  imagination,  and 
so  on),  and  also  overcomes  many  of  their  draw¬ 
backs. 

It  is  accepted  that  patients  with  ED  show  anxiety 
on  seeing  high-calorie  food  and  in  situations  in 
which  their  body  is  displayed  or  in  which  they 
come  into  contact  with  other  people^^.  In  our  de¬ 
sign  of  the  environments  we  incorporated  these 
two  variables  in  the  following  conditions:  Pres¬ 
ence  of  food  (No  food,  high-calorie  food  and  low- 
calorie  food)  and  presence  of  other  people  (No 
other  people  present  and  other  people  present). 
The  combination  of  the  two  variables  gives  rise 
to  a  repeated  measures  design  (2x3)  with  six 
experimental  conditions  or  virtual  environments: 
The  living-room  or  neutral  environment  (no  food 
and  no  other  people  present),  the  kitchen  with 
low-calorie  food  (low-calorie  food  and  no  other 
people  present),  the  kitchen  with  high-calorie 
food  (high-calorie  food  and  no  other  people  pre¬ 
sent),  the  restaurant  with  low-calorie  food  (low- 
calorie  food  and  other  people  present),  the  res¬ 
taurant  with  high-calorie  food  (high-calorie  food 
and  other  people  present)  and  the  swimming- 
pool  (no  food  and  other  people  present). 

This  study  aims  to  assess  the  effectiveness  of 
virtual  environments  to  provoke  emotional 
reactions  (anxiety  and  depressed  mood)  in 
patients  with  ED.  The  following  hypotheses  are 
considered:  the  first  one,  if  food  constitutes  an 
aversive,  anxiety-provoking  stimulus  for  people 
with  ED,  exposure  to  virtual  situations  in  which 
subjects  must  eat  different  types  of  food  (high 
or  low  calorie)  will  increase  their  level  of  anxiety 
and  depression:  the  second  one,  if  subjects 
with  ED  experience  higher  levels  of  stress  in 
social  situations  and  situations  that  involve 
scrutiny  by  others,  exposure  to  these  types  of 
situation  will  increase  their  level  of  anxiety  and 
depression. 

MATERIALS  AND  METHODS 

Procedure 

In  a  first  stage,  all  the  subjects  of  the  sample 
filled  two  self-report  questionnaires,  the  EDI-2 
(Eating  Disorders  Inventory)  and  the  STAI-T 
(Trait  Anxiety  Inventory).  Each  subject  was  then 
measured  and  weighed  individually  in  order  to 
calculate  their  body  mass  index  (BMI).  The 
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measurements  were  performed  after  the  ques¬ 
tionnaires  were  completed  to  avoid  the  possible 
influence  on  the  test  scores  of  anxiety  caused 
by  the  measuring  and  weighing.  Finally,  a  form 
was  filled  out  for  each  subject  specifying  their 
age,  weight,  height  and  BMI,  type  and  course  of 
the  disorder  and  severity  of  symptoms.  In  a  sec¬ 
ond  stage  the  six  virtual  environments  were  ran¬ 
domly  administered.  In  the  interval  between  the 
presentation  of  each  environment,  the  subject 
was  administered  the  STAI-S  (State  Anxiety 
Inventory)  and  the  CDB  (a  depression  scale). 
Both  tests  were  computerized  using  a  program 
that  integrates  them  in  the  sequence  of  virtual 
environments,  and  records  and  saves  the  data. 

Subjects 

The  sample  comprised  30  women  with  a  prior 
diagnosis  of  eating  disorder  (17  with  anorexia 
nerviosa,  11  with  bulimia  nerviosa  and  2  with 
non-specific  eating  disorder)  and  with  ages 
ranging  from  16  to  32  (mean  =  20.57  and  stan¬ 
dard  deviation  =  4.15).  The  patients  were  from 
hospital  centres  and  private  clinics  in  Barce¬ 
lona:  the  Hospital  Germans  Tries  i  Pujol,  the 
Hospital  de  Sant  Juan  de  Deu,  the  Clinica  La¬ 
bor  and  the  Centro  ABB. 


Situations 


Figure  1.  STAI-S  scores  obtained  after  visiting 
each  virtual  environment 


Instruments 

EDI-2  (Eating  Disorders  Inventory-2).  D.M.  Gar¬ 
ner.  Spanish  adaptation  by  S.  Corral,  M.  Gon¬ 
zalez,  J.  Pereha  &  N.  Seisdedos  (1998):  self- 
report  questionnaire  for  the  evaluation  of  symp¬ 
toms  that  normally  accompany  anorexia  ner¬ 
vosa  and  bulimia  nervosa.  The  questionnaire 
offers  scores  on  1 1  scales:  Drive  for  Thinness, 
Bulimia,  Body  Dissatisfaction,  Ineffectiveness, 
Perfectionism,  Interpersonal  Distrust,  Interocep¬ 
tive  Awareness,  Maturity  Fears,  Asceticism, 
Impulse  Regulation  and  Social  Insecurity. 

STAI  (Stait-Trait  Anxiety  Inventory).  C.D.  Spiel- 
berger,  R.L.  Gorsuch  and  R.E.  Lushene.  Span¬ 
ish  adaptation  by  N.  Seisdedos  (1988):  anxiety 
questionnaire  comprising  two  separate  self- 
report  scales  that  measure  two  independent 
concepts  of  anxiety,  a)  as  a  state  and  b)  as  a 
trait. 

CDB  (The  Barcelona  Depression 
Questionnaire).  J.  Gutierrez-Maldonado  and  M. 
Mora-Bello  (2000):  Self-report  instrument  for 
measuring  variations  in  depressed  mood.  It 
comprises  23  items,  each  one  with  a  visual- 
analog  scale  with  an  adjective  describing  a 
mood  written  at  the  top:  subjects  mark  the  line 
corresponding  to  their  experience  of  this  mood, 
ranging  from  “I  don’t  feel  like  this  at  all  now”  to 
“I  feel  completely  like  this  now”.  The  items  were 
taken  from  the  diagnostic  criteria  for  major  de¬ 
pression  from  the  nosological  classification  of 
the  DSM-IV  and  were  presented  in  the  ques¬ 
tionnaire  in  the  same  order  as  the  criteria.  All 
the  items  are  formulated  in  the  same  direction. 

Virtual  environments:  six  virtual  environments 
(living-room,  kitchen  with  low-calorie  food, 
kitchen  with  high-calorie  food,  restaurant  with 
low-calorie  food,  restaurant  with  high-calorie 
food,  and  swimming-pool)  developed  using  the 
virtual  reality  software  3d  Studio  Max5.1  and 
programmed  using  the  Lingo  language  with  the 
Director  software. 

RESULTS 

Using  repeated  measures  analysis,  we  com¬ 
pared  subjects’  scores  on  the  STAI-S  and  the 
CDB  after  visiting  each  of  the  five  experimental 
environments  with  the  score  obtained  after  visit¬ 
ing  the  neutral  environment  (the  living-room). 
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The  results  showed  significantly  higher  levels  of 
state  anxiety  in  the  kitchen  with  high-calorie 
food  {F=13,120:  p  =  0,001),  the  restaurant  with 
high-calorie  food  (F  =  14,  954;  p  =  0,001)  and 
the  swimming-pool  (F  =  4,230;  p  =  0,049)  than 
in  the  neutral  environment  (figure  1). 

Analysing  the  scores  for  depression  obtained 
on  the  CDB,  significant  differences  again  ap¬ 
peared  between  the  high-calorie  food  environ¬ 
ments  (F  =  7.187;  p  =  0.012  in  the  kitchen  and 
F  =  5.933;  p  =  0.021  in  the  restaurant)  and  the 
neutral  environment.  In  the  high-calorie  food 
situations  patients  with  ED  showed  a  more  de¬ 
pressed  mood  (figure  2). 

Finally,  no  interaction  was  found  between  the 
variables  “food”  and  “people”.  There  were  sig¬ 
nificant  differences  only  on  the  level  of  state 
anxiety  (STAI-S)  comparing  low-calorie  and 
high-calorie  food  environments  (F  =  15,262;  p  = 
0,001).  No  differences  were  found  between  en¬ 
vironments  with  people  and  those  without,  or 
when  analysing  the  combined  effect  of  these 
two  variables.  The  same  pattern  of  results  ap¬ 
pears  on  analysing  the  results  of  the  CDB:  the 
only  effect  is  exerted  by  food. 


Situations 


Figure  2.  CDB  scores  obtained  after  visiting 
each  virtual  environment 


DISCUSSION 

According  to  the  first  hypothesis,  virtual  envi¬ 
ronments  in  which  subjects  have  to  eat  high- 
calorie  food  (the  kitchen  and  restaurant  with 
high-calorie  food)  caused  significant  increases 
in  the  levels  of  anxiety  and  more  depressed 
mood  compared  with  the  neutral  situation  (the 
living-room).  Subjects  with  ED  show  negative 
reactivity  to  high  calorie  foods  due  to  their  fear 
of  gaining  weight.  Other  authors  as  Heilburn 
and  Flodin  (1989)  have  already  stressed  the 
aversive  effect  that  food  has  on  patients  with 
ED. 

In  reference  to  the  second  hypothesis,  we 
aimed  to  determine  whether  exposure  to  virtual 
environments  that  simulate  social  situations 
increases  the  level  of  state  anxiety  and  produce 
a  more  depressed  mood  in  people  with  ED. 
Only  the  swimming-pool  situation  significantly 
increases  the  level  of  anxiety  compared  with 
the  neutral  environment.  The  high  level  of  anxi¬ 
ety  shown  after  visiting  this  virtual  environment 
reflects  the  importance  that  patients  with  eating 
disorders  attribute  to  their  body  image  and  to 
the  evaluation  that  others  may  make  of  it.  In  this 
environment  subjects  wear  bathing-suits  and 
are  exposed  to  attractive  young  people  also 
wearing  bathing-suits,  so  aspects  such  as  body 
dissatisfaction  and  negative  social  comparison 
contribute  to  raise  their  level  of  anxiety.  Thomp¬ 
son  and  Chad  (2002)  affirm  that  subjects  with 
negative  perception  and  thoughts  regarding 
their  body  shape  are  usually  worried  about  how 
others  see  them;  this  is  clearly  the  case  of  pa¬ 
tients  with  ED.  The  authors  add  that  these  sub¬ 
jects  may  try  to  avoid  situations  in  which  their 
body  is  exposed  to  others,  as  is  the  case  of  the 
swimming-pool. 

One  of  the  aims  of  the  design  of  the  virtual  envi¬ 
ronments  was  to  make  the  effects  of  food  and 
people  additive,  so  we  expected  the  restaurant 
high-calorie  environment  to  provoke  the  highest 
levels  of  reactivity  in  subjects  because  of  the 
presence  of  people  and  the  presence  of  high- 
calorie  food  at  the  same  time.  However,  though 
there  was  no  interaction  between  the  variables, 
the  presence  or  absence  of  people  had  no  sig¬ 
nificant  effect  in  the  restaurant  situation.  Food 
was  the  only  effect  that  was  significant  in  all 
situations,  for  both  anxiety  and  depression.  This 
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means  that  the  subject’s  exposure  to  a  situation 
with  low  or  high  calorie  food  does  indeed  pro¬ 
duce  substantial  changes  in  their  level  of  anxi¬ 
ety  and  their  mood,  a  finding  that  appears  to 
confirm  our  first  hypothesis.  Nevertheless,  the 
second  hypothesis  is  not  verified. 

The  results  of  this  study  confirm  the  utility  of 
virtual  environments  as  instruments  capable  of 
provoking  emotional  reactions  in  patients  with 
eating  disorders,  but  they  show  too  that  it  is 
necessary  to  introduce  changes  to  improve  the 
effectiveness  of  some  of  the  developed  situa¬ 
tions.  Particularly,  the  interaction  with  the  appli¬ 
cation  should  be  increased,  especially  with  re¬ 
gard  to  the  avatars  representing  human  figures. 
In  contrast  to  the  food  stimuli,  which  the  subject 
can  eat,  in  none  of  the  environments  was  there 
any  interaction  between  the  subject  and  the 
other  people  present.  This  is  probably  why  the 
presence  or  absence  of  other  people  has  not 
produced  significant  changes  in  the  levels  of 
emotional  reactivity. 


CONCLUSION 

Virtual  reality  seems  to  be  a  valid  instrument 
particularly  useful  for  simulating  everyday  situa¬ 
tions  that  may  provoke  emotional  reactions 
such  as  anxiety  and  depression,  in  patients  with 
ED.  Virtual  environments  in  which  subjects  are 
obliged  to  ingest  high-calorie  food  provoke  the 
highest  levels  of  state  anxiety  and  depression. 
Nonetheless,  certain  changes  need  to  be  intro¬ 
duced  in  the  environments  in  order  to  increase 
the  impact  that  the  presence  of  people  has  on 
the  patients.  These  changes  should  aim  princi¬ 
pally  to  add  movement  and  sound  to  the  ava¬ 
tars  representing  human  figures  and  increase 
the  degree  of  interactivity  between  the  users 
and  the  virtual  environments. 
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Abstract;  Backgrounds:  Patients  with  schizophrenia  show  a  tendency  to  avoid  social  interaction  be¬ 
cause  they  feel  great  anxiety  during  their  social  interactions.  As  a  result,  they  have  trouble  in  making 
social  relationships.  This  trouble  is  largely  due  to  the  fact  that  these  patients  show  emotional  with¬ 
drawal  as  well  as  passive/apathetic  social  withdrawal.  Social  deficits  of  these  patients  have  been 
overcome  through  social  skills  training  programs,  which  provide  them  opportunities  to  experience 
various  social  situations  by  role  play  method.  Objectives:  This  study  was  conducted  in  order  to  inves¬ 
tigate  whether  an  interaction  with  a  virtual  avatar  can  evoke  patient’s  social  anxiety  and  the  relation¬ 
ship  between  patient’s  symptom  severity  and  social  anxiety  traits.  Method/Tools:  Fifteen  patients  with 
schizophrenia  and  fifteen  controls  were  recruited.  A  male  and  a  female  avatar  were  generated. 
2><2^3  (group,  avatar’s  gender  and  avatar’s  emotional  expression)  was  used  in  this  experiment.  Each 
avatar  has  three  emotional  expressions:  are  happy,  neutral  and  angry.  Subjects  performed  an  intro¬ 
duction  task  in  six  conditions  (gender  x  emotional  expression)  in  random  order.  The  task  was  com¬ 
posed  of  “approach”,  “listening  to  avatar’s  introduction”  and  “introduce  oneself  to  the  avatar”.  After  all 
six  tasks  were  performed,  subjects  completed  a  State-Trait  Anxiety  Inventory  (STAI)  questionnaire. 
In  addition,  patients’  symptom  severity  was  evaluated  using  the  Positive  and  Negative  Syndrome 
Scale  (PANSS).  Results:  The  social  anxiety  level  to  avatar’s  emotional  representation  was  signifi¬ 
cantly  different  in  the  two  group  (p<.001).  In  the  control  group,  the  level  was  lowest  when  subjects 
coped  with  an  avatar  expressing  “happy,  ”  while  the  highest  anxiety  level  was  shown  when  they  faced 
an  avatar  expressing  “angry.”  However,  in  the  patient  group,  the  level  difference  was  not  significant 
between  “neutral”  and  “happy”  avatar.  Patients  only  showed  a  significantly  high  anxiety  level  when 
they  experience  the  “angry”  avatar  condition  compared  to  other  two  conditions.  In  a  correlation  analy¬ 
sis  between  patient’s  anxiety  level  and  their  symptom  severity,  social  anxiety  for  “happy”  and 
“neutral”  avatars  was  positively  correlated  with  the  negative  syndrome  of  PANSS  (happy  :  r=.539, 
p=.038,  neutral :  r=.533,  p=.041).  Particularly,  the  anxiety  level  for  “happy”  and  “neutral”  avatars  was 
positively  correlated  with  two  subscales  (N1:  blunted  affect,  N4:  passive/apathetic  social  withdrawal) 
of  the  negative  syndrome  of  PANSS  (N1.  happy:  r=.549,  p=.034/ neutral:  r=.536,  p=.039)  (N4.  happy: 
r=.536,  p=. 039/  neutral:  r=.658,  p=.008).  Conclusion:  Through  this  study,  we  discovered  that  virtual 
avatar  displaying  emotional  expression  can  cause  social  anxiety  in  patients  with  schizophrenia  and 
that  a  patient’s  symptom  severity  was  correlated  with  social  anxiety  level.  It  could  be  said,  from  these 
results,  that  the  more  severe  the  negative  symptoms  a  patient  has,  the  greater  social  anxiety  they  will 
feel.  This  might  be  due  to  their  passive,  apathetic  and  isolated  trait.  They  may  tend  to  recognize  so¬ 
cially  positive  emotion  as  fearful  stimuli.  This  inference  could  be  supported  by  the  relationship  be¬ 
tween  anxiety  and  subscales  of  negative  symptom  of  PANSS.  In  this  sense,  we  could  say  that  a  vir¬ 
tual  avatar  could  provide  an  opportunity  for  patients  to  experience  emotionally  induced  social  situa¬ 
tion.  In  addition,  it  could  be  used  for  training  patients  to  cope  effectively  by  experiencing  emotions 
close  to  reality  as  well  as  to  find  out  the  clinical  characteristics  related  to  patient’s  symptoms. 
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INTRODUCTION 

Since  it  seriously  affects  higher  mental  func¬ 
tions,  such  as  thinking,  feeling,  and  perceiving,^ 
schizophrenia  can  be  considered  one  of  the 
most  devastating  psychiatric  disorders.  The 
clinical  course  of  schizophrenic  patients  is  char¬ 
acterized  by  higher  rates  of  relapse,  disability 
and  suicide  when  anxiety  symptoms  are  con¬ 
currently  experienced.^'^  Although  social  anxi¬ 
ety  symptoms  are  common  in  patients  with 
schizophrenia,'’  they  often  goes  unrecognized, 
or  are  considered  to  be  part  and  parcel  of  the 
schizophrenic  symptomatology.® 

In  fact,  many  patients  with  schizophrenia  suffer 
from  anxiety  symptoms.®  As  a  result,  they  ac¬ 
tively  avoid  social  interactions  and  appear  to 
have  little  motivation  to  develop  social  relation¬ 
ships.  Social  interaction  is  often  very  anxiety 
provoking  and  leads  to  avoidance;  patients  of¬ 
ten  seek  to  escape  from  interactions  initiated  by 
others.®  Existent  research  suggests  that  schizo¬ 
phrenic  patients  are  particularly  sensitive  to 
conflict  and  criticism  and  will  withdraw  from  po¬ 
tential  conflict  situations  even  when  they  are 
being  taken  advantage  of  or  unjustly  accused  of 
things  they  have  not  done.^ 

In  order  to  treat  these  patients  with  schizophre¬ 
nia,  effective  psychological  treatment  studies 
for  social  anxiety  have  been  developed  and 
evaluated.®®  A  combined  program  including 
exposure  treatment  together  with  cognitive  ther¬ 
apy  delivered  in  a  group  format  has  had  signifi¬ 
cant  outcomes.’®'’ "cognitive  behavioral  group 
treatment  (CBGT)  for  social  anxiety  in  schizo¬ 
phrenia  was  demonstrated  to  be  effective  as  an 
adjunctive  treatment’®  The  important  factors  in 
the  established  therapeutic  techniques  are  ex¬ 


posure  to  an  anxiety  evoking  situation  (gradual 
confrontation  of  feared  event/situation)  and  to 
learning  coping  strategies  for  a  situations 
through  role-play  (behavioral  acting  out).  It  is 
very  important  that  patients  experience  social 
anxiety  in  treatment  or  in  evaluation  session. 
However,  the  existing  therapeutic  programs 
depend  on  patient’s  imagination  ability.’® 

Therefore,  virtual  reality  could  be  a  powerful 
tool  to  provide  patients  with  schizophrenia  suf¬ 
fering  from  social  anxiety  to  be  immersed  in  a 
specific  situation  and  feel  and  experienced  so¬ 
cial  anxiety.  Accordingly,  the  aims  of  this  study 
were  to  develop  a  virtual  avatar  that  could  inter¬ 
act  with  patients  with  schizophrenia,  and  to 
evaluate  whether  a  virtual  avatar  could  evoke 
patient’s  social  anxiety.  In  addition,  this  study 
was  conducted  in  order  to  investigate  the  rela¬ 
tionship  between  patient’s  symptom  severity 
and  social  anxiety  traits. 

MATERIALS  AND  METHODS 

Apparatus  for  this  study 

The  apparatus  for  this  study  was  composed  of 
a  personal  computer,  a  6  DOF  (Degrees  of 
Freedom)  position  sensor,  an  HMD  (Head 
Mounted  Display)  and  tracker.  It  could  provide 
a  user  an  immersive  virtual  environment.  The 
position  sensor  was  composed  of  a  transmitter 
and  receiver.  The  transmitter  worked  as  an  ori¬ 
gin  creating  a  surrounding  magnetic  around  it, 
in  order  for  the  receiver  to  detect  its  position 
and  orientation  from  the  transmitter  by  calculat¬ 
ing  strength  of  the  magnetic  field.  The  receiver 
attached  on  the  user’s  head  could  detect  the 
user’s  head  position  and  orientation  so  that  the 


Schizophrenia 

Normal 

Age 

28.69  ±  7.064 

25.1  ±  1.553 

Gender 

M : 10/F  :  5 

M : 9/F  :  6 

PANSS 

Negative  Syndrome 

Positive  Syndrome 

18.85  ±3.23 

19.78  ±4.66 

38.21  ±  5.80 

General  Psychopathology 

Table  1.  Demographic  data  of  participants 
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computer  could  render  a  virtual  environment 
according  to  the  user’s  head  position  and  ori¬ 
entation. 

Virtual  Environment  and  Virtual  avatars 
Six  avatars  were  designed  for  this  study.  They 
were  divided  into  two  categories,  by  emotion 
expression  (happy,  neutral,  and  angry)  and 
gender  (male  and  female).  The  arousal  and 
valence  level  to  the  avatars’  facial  expression 
was  matched  through  a  previous  study. 

Clinical  Experiment 

Subjects 

Fifteen  patients  with  schizophrenia  partici¬ 
pated  in  this  experiment.  They  were  inpatients 
in  the  Severance  Mental  Hospital  at  Yonsei 
University.  Fifteen  normal  control  subjects 
were  recruited  for  this  study.  The  age  (p=.222) 
and  gender  ratio  (p=.891)  of  subjects  were  not 
significantly  different  between  normal  and  pa¬ 
tient  group. 

Experimental  design 

2  (group)  x2  (avatar’s  gender)  x3  (avatar’s 
emotional  expressions)  mixed  AVOVA  experi¬ 
mental  design  was  used  for  this  experiment. 
Each  subject  experienced  all  6  tasks  In  ran¬ 
dom  order. 

VR  Task  description 

The  task  was  composed  of  “approach,” 
“listening  to  avatar’s  introduction,”  and 
“introduce  oneself  to  the  avatar”  scenarios.  A 
subject  was  instructed  to  walk  up  to  an  avatar 
until  he  or  she  felt  comfortable  to  talk  with  the 
avatar.  The  subject  was  asked  to  say  “hello” 
first.  In  response,  the  avatar’s  introduction  was 
played.  At  the  end  of  avatar’s  introduction,  the 
avatar  asked  for  subject’s  introduction.  Then, 
the  subject  was  to  introduce  him/herself  to  the 
avatar  according  to  his  feelings  towards  the 
avatar  at  that  time. 

Experimental  Procedure 
When  a  subject  entered  the  experimental 
room,  the  subject  completed  a  demographic 
information  sheet  and  a  consent  form.  Subse¬ 
quently,  s/he  was  given  a  brief  introduction  to 
the  VR  devices  (HMD  and  tracker)  and  s/he 
was  fitted  into  the  equipment  and  positioned 
about  2.5  meter  from  an  avatar.  After  the  sub¬ 


ject  was  positioned,  the  position  sensor  was 
calibrated  so  that  all  subjects  could  see  the 
same  direction  in  virtual  environment  before  the 
experiment  started.  Then,  a  practice  task  was 
performed  prior  to  the  six  main  tasks.  An  avatar 
expressing  “neutral”  was  provided  as  a  practice 
session  so  that  the  subject  could  try  to  ap¬ 
proach  and  look  around.  After  all  six  main  tasks 
were  performed,  the  subject  filled  out  the  social 
anxiety  questionnaire  while  seeing  the  same 
avatar  again  in  order  to  recall  his  or  her  feelings 
during  the  task. 

Measurements 


*  Social  Anxiety  Questionnaire 


The  social  anxiety  questionnaire  data  was  ac¬ 
quired.  It  was  initially  developed  by  Spielberger 
et  al.^^  and  translated  into  Korean  and  vaiidat- 
ed^®  It  is  a  self-report  assessment  device  that  is 
composed  of  questions  reflecting  state  anxiety 
and  trait  anxiety.  According  to  Spielberger, 
state  anxiety  reflects  a  “transitory  emotional 
state  or  condition  of  the  human  organism  that  is 
characterized  by  subjective,  consciously  per¬ 
ceived  feelings  of  tension  and  apprehension, 
and  heightened  autonomic  nervous  system  ac¬ 
tivity.”  Therefore,  state  anxiety  may  fluctuate 
over  time  and  can  vary  in  intensity.  In  contrast, 
trait  anxiety  denotes  “relatively  stable  individual 
differences  in  anxiety  proneness.”  This  study 


Figure  1.  Comparison  of  social  anxiety  be¬ 
tween  patient  group  and  normal  group  at  each 
avatar’s  emotional  representation.  (*  represents 
the  significant  difference  at  the  0.05  level) 
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excluded  the  measurement  regarding  the  trait 
anxiety,  because  state  anxiety  would  better  rep¬ 
resent  social  anxiety  induced  by  experiencing 
of  emotional  virtual  environment  which  was  of 
interest  in  this  study. 

RESULTS 

The  main  effect  of  the  avatar’s  emotional  ex¬ 
pression  was  observed  in  both  group  (p<.001), 
while  a  main  effect  towards  the  avatar's  gender 
was  not  observed. 

An  interaction  effect  between  avatar’s  express¬ 
ing  emotion  and  the  subject  group  was  ob¬ 
served  (p=.007).  It  means  that  social  anxiety 
level  to  avatar’s  expressing  emotion  showed  a 
significantly  different  pattern  between  the  two 
groups.  In  the  control  group,  the  level  was  the 
lowest  when  they  faced  an  avatar  expressing 
“happy”,  and  the  highest  when  they  met  an  ava¬ 
tar  expressing  “angry”.  However,  members  of 
the  patient  group  did  not  show  any  anxiety  level 
difference  between  “happy”  and  “neutral”  ava¬ 
tar.  In  addition,  they  only  showed  a  significantly 
higher  social  anxiety  level  when  they  experi¬ 
enced  an  avatar  expressing  “angry”  compared 
to  other  emotional  conditions. 

In  short,  normal  control  subjects  felt  less  social 
anxiety  in  happy  and  neutral  condition  than  pa¬ 
tients  did  (happy:  p=.016/  neutral:  p=.332), 
while  normal  subjects  felt  more  social  anxiety  in 
angry  emotion  than  patients  did  (angry: 
p=.490). 

In  order  to  learn  how  a  patient’s  symptom  se¬ 
verity  affects  their  social  anxiety  level,  a  corre¬ 
lation  analysis  was  performed  between  PANSS 


and  their  social  anxiety  level.  As  a  result,  the 
negative  syndrome  score  of  PANSS  was  posi¬ 
tively  correlated  with  social  anxiety  on  “happy” 
and  “neutral”  avatars  (happy:  r=.539,  p=.038, 
neutral:  r=.533,  p=.041). 

In  particular,  a  significant  positive  correlation 
was  found  between  the  social  anxiety  level 
evoked  by  “happy”  and  “neutral”  avatars  and 
two  subscales  (N1:  blunted  affect,  N4:  passive/ 
apathetic  social  withdrawal)  of  the  negative 
syndrome  of  PANSS  (N1.  happy:  r=.549, 
p=.034/  neutral:  r=.536,  p=.039  ;  N4.  happy: 
r=.536,  p=.039/  neutral:  r=.658,  p=.008),  but 
there  were  no  correlations  with  other  subscales, 
such  as  positive  syndrome  and  general  psycho¬ 
pathology  score. 

DISCUSSION 

Schizophrenia  is  a  multi-dimensional  disorder 
and  necessitates  a  comprehensive  treatment 
approach. Social  anxiety  symptom  are  de¬ 
scribed  especially  frequently  by  patients  with 
schizophrenia  in  their  clinical  course.^'*  Patients 
tend  to  avoid  social  interactions  and  appear  to 
have  little  motivation  to  develop  social  relation¬ 
ships  because  social  interaction  often  is  very 
anxiety  provoking.®  This  study  explored 
whether  an  interaction  with  virtual  avatar  can 
provoke  patient’s  social  anxiety  and  investi¬ 
gated  the  relationship  between  patient’  symp¬ 
toms  severity  and  social  anxiety. 

The  observed  result  of  a  variation  of  anxiety 
level  to  a  virtual  avatar’s  emotional  expression 
shows  that  an  interaction  with  a  virtual  avatar 
can  induce  social  anxiety  of  patients  with 
schizophrenia.  It  means  that  a  virtual  reality  or 


Positive  syndrome 

Negative  syndrome 

General 

Psychopathology 

happy 

-.020  (p=.944) 

.539  (p=.038)* 

.591  (p=,020)* 

neutral 

-.038  (p=.893) 

.533  (p=.041)’* 

.474  (p=.074) 

angry 

-.323  (p=.240) 

-.163  (p=.563) 

.013  (p=.964) 

Table  2.  The  correlations  between  the  social  anxiety  level  to  avatar’s  emotional  representation 
(happy,  neutral,  angry)  and  PANSS 


( ) :  p-value, 

*  :  correlation  is  significant  at  the  0.05  level  (2-tailed) 
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Social  anxiety  level 

neutral 

angry 

N1.  Blunted  affect 

.549*(p=.034) 

.536*(p=.039) 

.021(p=.942) 

N2.  Emotional  withdrawal 

.305(p=.269) 

.194(p=.490) 

-.304(p=.271) 

N3.  Poor  rapport 

.409(p=.130) 

.374(p=.170) 

-.335(p=.222) 

N4.  Passive/apathetic  social  with¬ 
drawal 

.670**(p=.006) 

.658**(p=.008) 

-.054(p=.849) 

N5.  Difficulty  in  abstract  thinking 

.315(p=.253) 

.318(p=.249) 

-.155(p=.581) 

N6.  Lack  of  spontaneity 
&  flow  of  conversation 

.347(p=.206) 

-.155(p=.581) 

-.166(p=.554) 

N7.  Stereotyped  thinking 

.294(p=.287) 

.320(p=.245) 

.015(p=.958) 

Table  3.  The  correlation  between  the  social  anxiety  level  to  avatar’s  emotional  representation  and 
the  subscales  in  negative  symptoms  of  PANSS 


( ) :  p-value, 

*  :  correlation  is  significant  at  the  0.05  level  (2-taiied) 
**  :  correlation  is  significant  at  the  0.01  level  (2-tailed) 


a  virtual  avatar  can  evoke  social  anxiety  of  pa¬ 
tients  with  schizophrenia  so  that  the  character¬ 
istics  of  their  social  anxiety  can  be  assessed  or 
they  can  be  trained  in  a  socially  stressful  situa¬ 
tion. 

The  positive  correlation  between  the  negative 
symptom  score  of  PANSS  and  social  anxiety 
level  shows  that  the  more  severe  negative 
symptoms  a  patients  has,  the  higher  the  social 
anxiety  that  they  experience.  The  negative 
symptoms  include  characteristics  of  avolition 
and  anergia,  a  generalized  lack  of  motivation, 
energy,  and  initiative;  and  anhedonia,  an  inabil¬ 
ity  to  experience  pleasure  and  positive  emo¬ 
tions.  Therefore,  the  result  can  support  the  fact 
that  the  patients  with  schizophrenia  avoid  social 
interaction  because  the  negative  symptom 
mostly  appear  to  reflect  a  diminution  or  loss  of 
normal  functions  and  this  could  amplify  the 
anxiety  in  a  social  situation.  However,  there 
was  no  correlation  between  anxiety  and  the 
negative  symptom  severity  in  “anger”  condition. 
It  can  be  explained  by  an  assumption  that  the 
avatar  expressing  “anger”  evokes  an  anxiety 
too  intense  to  be  influenced  by  the  negative 
symptoms. 

The  positive  correlation  between  subscales  of 
Negative  symptom  revealed  in  the  further 
analysis  can  explain  which  characteristics  in 
negative  symptom  influence  patient’s  social 


anxiety.  The  results  showed  that  blunted  affect 
and  passive/apathetic  social  withdrawal  charac¬ 
teristics  were  correlated  with  the  social  anxiety. 
Therefore,  patients  with  schizophrenia  may  feel 
anxious  in  social  interactions  because  of  their 
inability  to  perceive  and  respond  properly  to 
other’s  emotions. 

CONCLUSION 

Through  this  study,  we  found  out  that  virtual 
emotionally  expressive  avatars  can  cause  so¬ 
cial  anxiety  to  patients  with  schizophrenia  and 
that  patient’s  symptom  severity  was  correlated 
with  social  anxiety  during  coping  with  avatar. 
From  these  results,  it  could  be  said  that  the  se¬ 
verer  negative  symptoms  the  patient  has,  the 
higher  social  anxiety  they  feel.  This  might  be 
due  to  their  passive,  apathetic  and  isolated  trait. 
Therefore,  they  even  tend  to  recognize  socially 
positive  emotion  as  fearful  stimuli.  This  infer¬ 
ence  could  be  supported  by  the  relationship 
between  anxiety  and  subscales  of  negative 
symptom  of  PANSS.  In  this  sense,  we  could 
say  that  a  virtual  avatar  could  provide  an  oppor¬ 
tunity  for  patients  to  experience  emotion  in¬ 
duced  social  situation.  In  addition,  it  could  be 
used  for  training  them  to  cope  effectively  by 
experiencing  emotions  close  to  reality  as  well 
as  to  find  out  the  clinical  characteristics  related 
to  patient’s  symptoms. 
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Abstract;  Background/Problem;  An  impairment  in  social  skills  is  one  of  the  few  criteria  that  all  indi¬ 
viduals  diagnosed  with  schizophrenia  must  meet  Successful  social  skills  require  the  coordination  of 
many  abilities,  including  social  perception,  which  involves  the  decoding  and  interpretation  of  social 
cues  from  others.  Previous  studies  on  social  perception  in  schizophrenia  have  focused  on  interpret¬ 
ing  the  emotions  from  facial  expressions.  There  are  some  reports  showing  the  potential  of  virtual  re¬ 
ality  (VR)  in  social  skills  training.  This  study  developed  VR  for  assessing  the  social  perception  includ¬ 
ing  an  interpretation  of  non-facial  expressions.  In  addition,  the  suitability  of  VR  for  determining  a 
schizophrenia  patient’s  social  perception  ability  was  evaluated. 

Method/Tools;  For  an  effective  social  perception  test,  VR  consisting  various  situations  and  contents 
may  be  needed.  Therefore,  the  VR  Behavior  &  Facial  Data  Base  architecture  (VRBFDB)  was  used 
for  the  easy  and  fast  VR  composition.  The  contents  can  be  divided  into  two  areas  “interpretation  of 
the  relevant  cue  test"  and  “Emotion  Recognition  test.”  The  “Interpretation  of  the  relevant  cue  test”  is 
composed  of  the  recognition  of  a  non-verbal  social  situation  as  well  as  the  recognition  of  a  verbal 
social  situation,  and  the  “Emotion  Recognition  test”  is  composed  of  happy,  sad,  angry  and  surprise 
facial  expression  recognition. 

There  were  17  subjects  (12  males  and  5  females)  and  17  controls  (12  males  and  5  females).  This 
study  obtained  the  data  from  the  VR,  a  questionnaire,  and  the  subject’s  symptoms.  The  VR  data  in¬ 
cluded  the  participant’s  response  results,  reaction  time,  presence  and  computer  experience  .  The 
questionnaire  data  included  age,  education,  gender,  intelligence  (K-WAIS)  etc.  The  symptom  data 
was  measured  using  the  FANSS  (Fositive  and  Negative  Syndrome  Scales). 

Results/Conclusion:  According  to  the  results,  schizophrenia  subjects  perform  significantly  worse  on 
the  VR  test  than  normal  subjects,  and  some  parameters  the  correlated  with  previous  clinical  test. 


INTRODUCTION 

“Social  skills”  are  interpersonal  behaviors  that 
are  normative  and/or  socially  sanctioned.  They 
include  items  such  as  dress  and  behavior 
codes,  rules  about  what  to  say  and  what  not  to 
say,  and  stylistic  guidelines  about  the  expres¬ 
sion  of  affect,  sociai  reinforcement,  interper¬ 
sonal  distance  etc  [1],  Impairment  in  sociai  skills 
is  one  of  the  criteria  that  all  individuals  diag¬ 
nosed  with  schizophrenia  must  meet  [2].  These 
deficits  make  it  difficuit  for  many  clients  to  es¬ 
tablish  and  maintain  social  relationships,  fulfill 
social  roles  or  to  have  their  needs  met  [1]. 


Successful  social  competence  is  based  on  a 
distinct  set  of  component  skills  [3],  These  com¬ 
ponents  can  be  roughly  divided  into  two  broad 
sets:  expressive  skills  (speech  content,  paralin- 
guistic  features,  and  nonverbal  behavior)  and 
social  perception  skills  (interpretation  of  the 
relevant  cues,  emotion  recognition).  However, 
regardless  of  the  individual’s  ability  to  emit  so- 
ciaily  skillful  responses,  he  or  she  cannot  be 
effective  without  an  accurate  social  perception 
that  involves  the  decoding  and  interpretation  of 
various  social  cues  in  others  [1].  Previous  re¬ 
search  on  social  perception  in  schizophrenia 
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focused  on  the  interpretation  of  emotion  from 
the  facial  expressions.  Although  facial  expres¬ 
sion  is  admittedly  an  important  social  cue.  the 
restrictive  focus  on  the  interpretation  of  this  sin¬ 
gle  nonverbal  cue  was  too  narrow  given  the 
breadth  of  the  social  deficits  in  schizophrenia 
[41. 

There  are  many  social  cue  elements  that  sub¬ 
stantially  contribute  to  socially  skillful  behavior. 
And  social  cue  perception  is  the  most  funda¬ 
mental  requirements  for  accurate  social  percep¬ 
tion  [1].  These  social  cues  can  be  divided  into 
verbal  or  nonverbal  social  signals  [4].  Many 
chronic  clients  exhibit  poor  interpersonal  behav¬ 
ior  precisely  because  their  focus  of  attention  is 
primarily  internal,  and  is  only  intermittently  and 
selectively  directed  outward.  Another  aspect  of 
social  perception  involves  the  accurate  percep¬ 
tion  of  emotion.  Emotion  is  frequently  communi¬ 
cated  by  a  subtle  combination  of  verbal  and 
nonverbal  cues.  Given  that  the  emotion  states 
of  an  interpersonal  partner  are  critical  factors  in 
determining  the  appropriate  response,  a  so¬ 
cially  skilled  individual  must  be  able  to  ‘read’  the 
emotional  cues.  Such  personal  perception  and 
analysis  enhances  accurate  communication 
and  is  essential  for  the  effective  resolution  of 
conflict  and  distress  [1]. 


The  existing  social  perception  and  social  skill 
training  method  rely  upon  role-play,  either  acted 
out  or  observed  through  the  use  media  and  ma¬ 
terials  [5,  6].  Typically,  it  involves  the  use  of  a 
puppet  or  videotape  [7].  However,  despite  the 
success  of  some  social  skills  training  programs 
in  enhancing  skill  acquisition  within  a  training 
setting,  it  has  limitations  in  the  transfer  of  train¬ 
ing  and  the  generalization  of  skills  to  the  natural 
environment  [8].  Moreover,  it  does  not  focus  on 
social  perception,  which  is  a  basic  and  primitive 
aspect  of  social  skills. 

Recently,  there  have  been  some  reports  on  the 
potential  of  virtual  reality  (VR)  in  social  skills 
training  [7,  9,  10].  Virtual  environments  (VE) 
allow  the  users  to  interact  with  a  three- 
dimensional  computer-based  world  incorporat¬ 
ing  impressive  graphics  and  design.  The  users 
can  move  through  a  VE  in  real  time  using  a  joy¬ 
stick  or  mouse.  They  can  interact  with  the  ob¬ 
jects  in  a  scene  and  are  typically  represented 
as  a  humanoid  “avatar”  [9].  The  cardinal  feature 
of  VR  is  the  provision  of  a  sense  of  actual  pres¬ 
ence  in  the  stimulated  environment  [11,  12].  In 
addition,  complete  control  over  the  content  is 
possible,  and  the  performance  data  can  be 
stored  in  a  database.  VR  provides  patients  with 
added  motivation  by  adding  gaming  factors  in  a 
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Figure  1.  Hardware  for  virtual  reality 
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safe  virtual  environment  that  eliminates  the 
risks  caused  by  errors  [13]. 

This  study  developed  a  VR  Behavior  &  Facial 
Data  Base  architecture  (VRBFDB)  for  an  easy 
and  fast  VR  composition.  The  VTBFDB  was 
used  to  design  VR  social  perception  contents 
including  the  interpretation  of  verbal  cue  per¬ 
ception,  non-verbal  cue  perception,  and  the 
recognition  of  emotion.  It  was  hypothesized  that 
schizophrenia  subjects  would  perform  signifi¬ 
cantly  worse  on  the  VR  test  than  normal  sub¬ 
jects. 

MATERIALS  AND  METHODS 

System 

The  VR  system  consisted  of  a  Pentium  IV  PC, 
DirectX  3D  Accelerator  VGA  Card,  Projection 
Screen,  and  Joystick  (Dahoon  DHU-1500), 
which  could  be  used  as  an  easy  response  to 
allow  the  patients  to  perform  the  virtual  reality 
scenario  comfortably.  The  PC,  which  is  fitted 
with  a  3D  Accelerator  VGA  Card,  generates 
real-time  virtual  images  for  the  subject.  Figure  1 
shows  the  hardware  for  the  VR. 

VRDFDB 

A  VR  software  framework,  called  a  VR  Behav¬ 
ior  and  Facial  Data  Base  (VRBFRDB)  was  set 
up  to  provide  a  variety  of  VR  scenarios.  The 
VRBFDB  is  composed  of  an  avatar  database 
(14  avatars)  as  well  as  a  motion  and  facial  ex¬ 


pression  database.  The  motion  and  facial  DB 
includes  93  behaviors  and  7  facial  expressions. 
14  avatars  designed  by  the  3D  Max  Studio 
could  be  used  in  different  situations  by  applying 
a  different  texture  map,  which  affects  the  ava¬ 
tar’s  clothes,  hair  style,  and  skin  color.  Seven 
facial  expressions  were  implemented  using  a 
morphing  technology.  Primitive  facial  expres¬ 
sions  (happy,  sad,  angry,  smile,  surprising,  em¬ 
barrassing,  and  bored)  were  controlled  by  the 
parameters  to  express  complicated  and  subtle 
facial  expression.  Ninety-three  behaviors  are 
made  by  a  biped,  which  is  used  to  represent  the 
avatar’s  action.  Each  behavior  could  be  at¬ 
tached  to  any  avatar,  which  was  already  made, 
and  the  avatar’s  animation,  which  was  matched 
with  each  scenario,  could  be  made. 
According  to  the  VR  scenario,  the  avatars,  be¬ 
haviors,  and  facial  expressions,  which  are 
needed  in  the  scenario,  were  integrated  and 
then  exported  in  .Med  files.  The  necessary  en¬ 
vironment  in  the  scenario  was  exported  to 
a  .Map  file,  which  was  used  in  the  A6  Game 
Studio  Engine.  The  exported  avatar,  which  in¬ 
cluded  the  behaviors  and  facial  expression,  was 
merged  with  the  .Map  file  and  rendering  by  an 
A6  Game  Studio  rendering  engine.  Figure  2 
shows  the  VRBFDB  architecture. 

Virtual  environment  and  task 

VR  is  designed  to  assess  the  social  perception. 
During  the  experiment,  the  subjects  perform  VR 
and  he/she  chooses  whether  it  is  suitable  or 
unsuitable  (Likert  type  scale:  5  point).  The  VR  is 


Figure  2.  VR  Behavior  &  Facial  Data  Base  Architecture  (VRBFDB) 
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Interpretation  of  relevant  cue 

Nonverbal  social  cue  (8) 

-  Recognize  avatar’s  physical  gesture 

-  Recognize  traffic  signal 

-  Unusual  street  situation 

-  Suitable/Unsuitable  behavior  according  to  situation 

Verbal  social  cue  (6) 

-  Polite/rude  dialog 

-  Suitable/Unsuitable  greeting 

Emotional  Recognition 

Happy  situation  (6) 

-  Recognize  happy  facial  expression 

-  Recognize  happy  situation 

Sad  situation  (4) 

-  Recognize  sad  facial  expression 

-  Recognize  sad  situation 

Angry  situation  (6) 

-  Recognize  angry  facial  expression 

-  Recognize  angry  situation 

Table  1.  The  contents  of  the  VR  social  perception  tool. 


composed  of  an  ‘Interpretation  of  Relevant  Cue 
(IRC)’  and  ‘Emotional  Recognition  (ER)’.  Table 
1  shows  the  contents  of  the  VR.  The  IRC  is 
composed  nonverbal  and  verbal  social  cues. 
The  ER  is  composed  a  happy  situation,  a  sad 
situation  and  an  angry  situation.  In  the  nonver¬ 
bal  social  cue  perception,  the  physical  gesture 
and  social  situation  was  selected  because  they 
can  be  used  as  virtual  ‘avatar’  and  virtual 
‘context’.  In  addition,  the  VR  contents  for  verbal 
social  cue  were  composed  of  a  basic  verbal 
component  (for  example  suitable  or  unsuitable 
greeting  and  polite  or  rude  expression).  In  the 
emotion  recognition,  happy,  sad,  angry  emotion 
recognition  contents  were  designed.  During  the 
emotional  recognition  contents,  the  patients 
must  recognize  the  emotion  considering  the 
avatar’s  facial  expression  and  context. 


EXPERIMENT  (SUBJECTS,  PROCEDURES, 
MEASUREMENT) 

Subjects. 

The  subjects  consisted  of  17  inpatients  (12 
males  and  5  females)  diagnosed  with  schizo¬ 
phrenia,  and  17  control  subjects  (12  males  and 
5  females)  (Table  2).  The  patients’  mean  total 
PANSS  (Positive  and  Negative  Syndrome 
Scale)  was  77.12  (SD  8.43),  the  positive  and 
negative  PANSS  means  were  18.00  (SD  2.83) 
and  20.18  (SD  3.26)  respectively. 

There  were  no  significant  differences  between 
the  patient  and  control  groups.  The  score  on  the 
patient’s  computer  using  the  experience  was 
checked  as  (1)  very  strange,  (2)  strange,  (3) 
common,  (4)  experienced,  and  (5)  very  experi¬ 
enced. 


Patient  group  (n=17) 

Control  group  (n=17) 

Age  (years) 

30.41  ±  5.36 

30.05  ±  6.07 

Sex  (M:F) 

12:5 

12  :  5 

Computer-using  experience 

3.41  ±  1.23 

3.82  ±  1.13 

Table  2.  Subject 
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Patient  group 
(Schizophrenia) 

Control  group 
(Normal) 

Interpretation  of  relevant  cue 

3.49  ±  0.75** 

4.20  ±  0.40 

Emotional  Recognition 

3.32  ±  0.35** 

3.92  ±  0.32 

Nonverbal  social  cue 

3.44  ±  0.97** 

4.24  ±  0.42 

Verbal  social  cue 

3.54  ±  0.63** 

4.17  ±0.54 

Happy  situation 

3.49  ±  0.59** 

4.38  ±  0.29 

Sad  situation 

3.34  ±  0.62** 

3.93  ±  0.57 

Angry  situation 

3.13  ±0.31* 

3.47  ±  0.49 

*  p<0.05,  **p<0.01  between  patients  group  and  control  group 


Table  3.  Results  of  the  Parameters  from  the  virtual  Environment 


Procedure. 

Prior  to  the  experiment,  the  subjects  were 
asked  to  complete  the  form  containing  the  pa¬ 
tient’s  name,  age,  education  and  computer  ex¬ 
perience,  and  were  examined  using  the  psycho¬ 
logical  tests  (ITQ,  K-WAIS).  In  addition  to  this 
test,  the  patients  were  examined  using  the  Posi¬ 
tive  and  Negative  Syndrome  Scale  (PANSS). 
During  the  experiments,  the  subjects  first  read 
the  instruments  and  completed  a  pretest  to  use 
the  joystick.  In  the  VR  tasks,  the  subjects  ob¬ 
served  randomized  VR  contents  and  response 
according  to  their  feelings.  After  the  VR  tasks, 
the  subjects  were  asked  to  complete  the  form 
PQ  (Presence  Questionnaire)  and  VRQ  (Virtual 
Reality  Questionnaire).  The  form  contains  esti¬ 


mations  about  their  presence,  experimental  en¬ 
vironment  and  virtual  reality. 

Measurements. 

This  system  measured  various  parameters 
while  the  subject  experienced  the  VR  system. 
As  shown  Table  3,  the  interpretation  of  the  rele¬ 
vant  cue,  the  emotion  recognition,  the  nonver¬ 
bal  social  cue,  the  verbal  social  cue,  the  happy 
situation,  the  sad  situation,  and  the  angry  situa¬ 
tion  were  measured.  The  interpretation  of  the 
relevant  cue  and  emotion  recognition  were  cal¬ 
culated  according  to  their  subcomponent  in  ta¬ 
ble  1. 


Response  performance  Response  time 


r 

.w. 

o  Time  Nonverbal 

Time  Verbal 
I 

D  Time  Happy  Sit. 

q  Time  Sad  Sit. 
Time  Angry  Sit 


Figure  3.  The  mean  comparison  between  the  patient  and  normal  group,  (a)  VR  response  compari¬ 
son.  (b)  VR  response  time  comparison. 
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RESULTS 

According  to  Table  3,  the  VR  parameter  in  the 
patient  and  control  groups  was  significantly  dif¬ 
ferent  (p<0.05  or  p<0.01).  In  addition,  there  was 
no  significant  deference  between  the  groups 
(patient  and  control  group;  see  Table  1).  The 
seven  parameters  in  Table  3  show  a  significant 
difference  between  the  patient  and  normal 
groups  when  using  this  program  as  an  assess¬ 
ment  tool. 

The  results  given  in  Figure  3  show  a  mean 
comparison  between  the  patient  and  normal 
groups.  Figure  3a  shows  that  the  schizophrenia 
group’s  social  perception  ability  was  signifi¬ 
cantly  poorer  than  the  normal  group.  In  particu¬ 
lar,  the  schizophrenia  group’s  nonverbal  social 
cue  perception  ability  was  poorer  than  the  ver¬ 
bal  social  cue  perception  ability.  In  the  normal 
group,  the  nonverbal  social  cue  perception  abil¬ 
ity  was  greater  than  the  verbal  social  cue  per¬ 
ception  ability.  Figure  3b  shows  the  response 
time  of  the  schizophrenia  patient  occurred  later 
than  that  of  the  normal  group.  However,  there 
was  no  statistically  significant  difference  and  no 
tendency  difference. 

In  order  to  assess  the  relationship  between  the 
social  perception  ability  and  symptoms,  the 
Spearman’s  correlation  between  the  perform¬ 
ance  on  VR  and  PANSS  was  computed.  There 
is  no  significant  difference  between  the  social 


perception  component  and  the  PANSS  total 
scores  (positive,  negative  and  general  score). 
The  result  shown  in  Table  4  shows  a  correlation 
between  the  social  perception  component  and 
the  PANSS  individual  scores.  However,  more 
conservative  criteria  are  needed  when  interpret¬ 
ing  the  correlation  in  the  PANSS  individual 
score.  As  the  correlation  was  significant  at 
the  .01  level,  the  verbal  cue  perception 
(Auditory)  showed  a  negative  correlation  with 
delusions,  the  recognition  of  a  happy  facial  ex¬ 
pression  had  a  negative  correlation  with  con¬ 
ceptual  disorganization,  and  the  recognition  of 
an  angry  facial  expression  had  a  negative  cor¬ 
relation  with  stereotyped  thinking. 

DISCUSSION 

VR  was  developed  using  the  VRDFDB  for  a  fast 
and  easy  composition.  This  study  examined  the 
social  perception  between  a  schizophrenia 
group  and  a  normal  group  using  the  developed 
VR. 

There  are  many  advantages  in  VRDFDB.  First, 
it  can  be  fast  developing.  The  user  only  selects 
the  behavior  and  facial  expression  according  to 
a  suitable  sequence  in  a  predefined  Data  Base. 
The  fast  development  may  be  an  essential 
component  in  a  VR  Social  perception  tool  be¬ 
cause  social  perception  requires  various  situa¬ 
tions.  Second,  VRDFDB  can  be  easily  modified. 
Previous  VR  systems  were  difficult  to  modify 


PANSS  (Positive  and  Negative  Syndrome 

Scale) 

Delusions 

Conceptual 

Disorganiza¬ 

tion 

Suspicious¬ 

ness 

/Persecution 

Stereotyped 

thinking 

Behavior  Cue  perception 

.107 

-.010 

.504* 

-.028 

.684 

.968 

.039 

.914 

Verbal  Cue  perception 

-.659** 

-.497* 

.113 

-.166 

(Auditory) 

.004 

.042 

.665 

.525 

Recognize  happy  facial 

-.388 

-.638** 

.253 

-.435 

.124 

.006 

.327 

.081 

Recognize  sad  facial 

-.008 

.092 

-.400 

.976 

.727 

.111 

Recognize  angry  facial 

-.293 

.021 

-.634** 

.254 

1  .011 

.935 

.006 

*.  Correlation  is  significant  at  the  .05  level 
**.  Correlation  is  significant  at  the  .01  level 


Table  4.  Correlation  analysis  with  PANSS 
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during  the  developing  period.  However, 
VRDFDB  can  be  modified  easy  by  simply  se¬ 
lecting  another  behavior  and  a  facial  change 
parameter.  On  the  other  hand,  VRDFDB  has 
some  problem.  First,  it  has  limited  behaviors 
and  facial  expressions.  In  a  one  time  DB  com¬ 
position,  there  are  14  avatars  with  93  behaviors 
and  7  types  of  facial  expressions.  Despite  this, 
there  is  still  a  large  number.  However,  more 
avatars  and  behaviors  are  needed  for  sufficient 
social  perception  content.  Second,  it  is  still  a 
programmer  level  architecture.  Indeed,  only 
programmer  can  develop  VR  using  our  archi¬ 
tecture.  Ideally,  everyone  including  clinicians 
could  make  VR,  but  in  view  of  current  VR  re¬ 
search  this  is  impossible  until  now. 

The  results  of  the  VR  social  perception  tasks 
require  more  research  in  clinical  aspect.  How¬ 
ever,  some  results  are  statistically  significant  or 
concur  with  previous  reports.  First,  according  to 
the  result  shown  in  figure  3  and  table  3,  the  so¬ 
cial  perception  abilities  in  schizophrenia  and 
normal  groups  were  significant  different.  This 
result  confirms  the  previous  research  using  VR 
that  the  schizophrenia  patient's  social  percep¬ 
tion  ability  is  worse  than  normal  subjects  [14]. 
Second,  contrary  to  the  tendency  of  a  normal 
performance  task  between  nonverbal  and  ver¬ 
bal  social  perception,  there  was  a  reverse  ten¬ 
dency  within  the  schizophrenia  patient  group, 
which  partially  correlated  with  Toomey’s  re¬ 
search  [15].  Third,  although  a  correlation  be¬ 
tween  Positive  and  Negative  Syndrome  Scale 
(PANSS)  and  virtual  reality  parameters  was 
found,  more  research  will  be  needed  to  confirm 
the  clinical  significance. 

This  paper  used  VRDFDB  for  the  fast  and  easy 
construction  of  a  virtual  environment,  and 
tested  the  possibility  of  a  social  perception  sys¬ 
tem  using  VR.  Although  a  more  profound  dis¬ 
cussion  about  the  virtual  reality  parameters  will 
be  needed  to  explain  the  clinical  result,  a  VR 
system  can  be  used  to  assess  a  schizophrenia 
patient’s  social  cue  perception  ability  by  com¬ 
paring  the  performance  between  a  normal  and 
schizophrenia  group  in  a  given  virtual  reality 
task.  In  future,  VR  will  be  used  to  train  and  as¬ 
sess  the  social  problem  solving  ability  or  social 
skills  on  the  whole. 
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Abstracts:  A  virtual  avatar  has  been  used  for  various  applications  that  require  communication 
among  persons,  or  to  train  or  educate  peopie  by  demonstrating  human-iike  behavior.  Recently,  much 
research  has  shown  an  enhancement  of  virtual  avatar  technology,  with  the  avatar  perceived  as  a  reai 
person.  As  a  result,  the  technology  could  begin  to  be  used  for  observing  human  behavior  to  a  virtuat 
avatar.  This  paper  concerns  whether  a  virtual  avatar  could  acquire  patients’  behavioral  characteris¬ 
tics  during  experiencing  a  virtual  task  which  is  composed  of  approaching  and  conducting  a  short  con¬ 
versation.  For  this,  we  designed  a  virtual  avatar  that  was  standing  in  a  virtuai  room,  with  eleven 
schizophrenic  patients  assigned  the  task  of  approaching  the  virtual  avatar,  initiating  a  conversation, 
and  providing  answers  to  the  avatar’s  questions.  To  measure  behavioral  parameters  in  the  virtuat 
environment,  we  acquired  the  interpersonal  distance  and  the  verbal  response  time.  In  addition,  we 
rated  patients  on  the  Positive  and  Negative  Syndrome  Scaie  (PANSS)  in  order  to  investigate  a  reia- 
tionship  between  patients’  symptomatic  characteristic  and  behavioral  parameters.  Results  of  this 
study  revealed  that  the  interpersonal  distance  was  negativeiy  correlated  with  the  negative  syndrome 
scaie,  which  is  a  subscale  of  PANSS  (r=-0.687,  p=0.02).  By  contrast,  the  verbal  response  time  was 
not  correlated  with  any  other  subscale  of  PANSS.  However,  after  analyzing  this  variable  with  sub- 
items  of  the  negative  syndrome  of  PANSS,  two  positive  correlations  were  found:  one  with  blunted 
affect  (r=.638,  p=.035)  and  the  other  with  poor  rapport  (r=.615,  p=.044).  The  negative  correlation  be¬ 
tween  the  distance  and  negative  symptom  severity  observed  in  this  study  is  consistent  with  studies 
that  investigated  the  reiationship  between  schizophrenic  patients’  interpersonal  distance  and  their 
symptoms.  The  positive  correlation  between  the  verbal  response  time  acquired  and  subscales  in 
PANSS  in  this  study  coutd  be  explained  by  the  definition  of  these  subscales.  Therefore,  this  positive 
correlation  means  that  the  less  a  patient’s  emotional  response,  intimacy,  and  relationship  making 
with  a  virtuai  avatar,  the  stower  they  answer  the  avatar’s  question.  Inferring  from  these  resuits,  we 
conctude  that  the  virtual  avatar  could  be  perceived  as  a  real  human  by  schizophrenic  patients,  the 
avatar  could  facilitate  the  schizophrenic  patients’  behaviorat  characteristics,  and  the  avatar  could  be 
used  as  a  tool  for  assessing  the  behavioral  characteristics  of  patients  with  schizophrenia. 


INTRODUCTION 

Virtual  Reality  (VR)  has  the  potential  to  provide 
a  realistic  three-dimensional  world  generated  by 
computer  graphics,  with  which  the  user  can  in¬ 
teract,  so  that  he  or  she  can  navigate  within  and 
manage  the  virtual  world  and  obtain  computer¬ 
ized  objective  scores.  Due  to  its  capacity  to  pro¬ 
vide  realistic  three-dimensional  environments  to 
users,  VR  has  already  been  applied  to  many 
fields  such  as  industry,  the  military,  entertain¬ 
ment  and  medicine.  In  particular,  many  medical 
applications  have  emerged  because  of  its  ability 


to  provide  a  realistic,  interactive,  immersive, 
and  safe  environment;  this  complements  the 
fact  that  it  provides  a  flexible  and  controlled  ex¬ 
perimental  and  therapeutic  environment.  It  has 
been  applied  to  several  types  of  mental  disor¬ 
ders  such  as  illness,’"^  and  attention  deficit  dis¬ 
orders  (ADD)’  '*  for  assessing  patients’  cognitive 
functions^  and  for  providing  training  for  stroke 
patients’  activities  of  daily  living  (ADL).®  VR  has 
also  been  used  to  assess  and  investigate  hu¬ 
man  characteristics  in  three-dimensional  envi- 
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ronments  such  as  for  navigation/  and  spatial 
memory.® 

Moreover,  recent  technological  advances  in 
avatars  enable  computer-generated  entities  to 
mimic  both  the  appearance  and  behaviors  of 
humans.  Thus,  Virtual  Environments  (VEs)  can 
be  more  realistic  and  social,  because  VEs  can 
be  populated  by  representations  of  people  as 
well  as  objects.  Recent  improvements  in 
graphic  and  animation  technology  have  made  it 
possible  for  the  avatars  used  in  these  visual 
images  to  appear  increasingly  human-like. 
Studies  have  shown  that  people  report  feeling 
some  level  of  presence  in  almost  all  mediated 
environments  and  even  respond  socially  to 
computer-generated  human-like  entities.® 

In  addition  to  these  results,  several  studies 
have  shown  that  human  behavioral  characteris¬ 
tics,  such  as  proximity  and  social  influence 
could  be  investigated  using  a  virtual  avatar  or  a 
social  situation  populated  with  several  ava- 
tars.^°'^^  These  studies  support  the  hypothesis 
that  a  VE  populated  with  avatars  could  influ¬ 
ence  human  behavior  and  emotion. 

From  these  studies  showing  the  capacity  and 
potential  of  VR,  we  can  predict  that  VR  could 
also  be  useful  for  investigating  the  interpersonal 
behavior  of  patients  with  mental  illness.  In  par¬ 
ticular,  we  are  interested  in  applications  to 
schizophrenia  because  it  is  one  of  the  most 
devastating  psychiatric  disorders,  as  it  seriously 
affects  the  higher  mental  functions,  such  as 
thinking,  feeling  and  perceiving.’®  The  Diagnos¬ 
tic  and  Statistical  Manual  of  Mental  Disorders, 
fourth  edition  (DSM-IV),  states  that  schizophre¬ 
nia  involves  problems  in  one  or  more  major  ar¬ 
eas  of  functioning  (e.g.,  interpersonal  relations, 
work  or  self-care).’®  It  is  also  characterized  by 
symptoms  such  as  hallucinations  or  disorgan¬ 
ized  thinking,  the  loss  of  goal-directed  behav¬ 
iors,  and  deterioration  in  social  role  functioning. 
Due  to  these  deficits,  patients  with  schizophre¬ 
nia  have  a  negative  effect  on  their  partners  in 
interpersonal  or  social  interaction  and  they  suf¬ 
fer  when  living  with  others.  Although  investigat¬ 
ing  the  interpersonal  behavioral  characteristics 
of  patients  with  schizophrenia  is  necessary  to 
understand  and  treat  them,  currently  only  sub¬ 
jective  materials  such  as  videotaping  or  clinical 
observation  are  available  for  assessing  pa¬ 


tients’  interpersonal  behavior  characteristics 
during  an  interaction.  The  problems  of  those 
methods  for  assessing  the  patient’s  behavior 
concern  the  difficulty  in  establishing  a  controlled 
experimental  situation  and  observing  behavior 
consistently. 

Due  to  these  needs,  in  this  study,  we  aimed  to 
develop  a  method  for  assessing  the  interper¬ 
sonal  behavioral  characteristics  of  patients  with 
schizophrenia  using  VR  technology.  VR  tech¬ 
nology  has  many  advantages  in  providing  a 
realistic  environment,  and  allows  us  to  investi¬ 
gate  how  schizophrenic  patients  perceive  and 
react  to  a  virtual  avatar.  It  also  allows  an  inves¬ 
tigation  of  whether  interpersonal  behavior  to¬ 
ward  a  virtual  avatar  is  similar  to  behavior  in 
response  to  a  human. 

For  accomplishing  the  goal  of  this  study,  a  vir¬ 
tual  room,  in  which  a  virtual  avatar  is  standing, 
was  designed  and  a  short  conversation  task 
was  assigned  to  the  patients.  And,  as  objective 
parameters  from  patients’  virtual  experience, 
the  interpersonal  distance  and  the  verbal  re¬ 
sponse  time  were  chosen  because  they  are 
representative  factors  to  represent  a  relation¬ 
ship  between  people  and  play  an  important  role 
in  conversation  as  well  as  social  life.’^  And, 
PANSS  of  the  patients  were  also  acquired  in 
order  to  investigate  the  relationship  between 
their  symptom  and  the  behavioral  parameters. 
Through  the  investigation  of  the  relationships 
among  these  parameters,  we  expected  that  the 
patients  react  to  the  virtual  avatar  in  the  same 
way  as  to  human  and  the  virtual  environment 
and  the  virtual  avatar  could  be  a  tool  for  as¬ 
sessing  the  interpersonal  behavior  characteris¬ 
tics  of  the  patients  with  schizophrenia. 

MATERIALS  AND  METHOD 

Subjects 

We  recruited  1 1  patients,  5  male  and  6  female, 
diagnosed  with  schizophrenia,  for  this  experi¬ 
ment.  They  were  inpatients  at  the  Severance 
Mental  Health  Hospital  in  Korea  and  were  able 
to  perform  a  task  and  to  control  an  interface 
such  as  a  joystick.  Their  mean  age  was  29.54  ± 
8.95  years  (mean  ±  SD)  and  their  PANSS  score 
were  21.81  ±  4.62  (mean  ±  SD)  for  negative 
symptoms,  19.45  ±3.67  (mean  ±  SD)  for  posi- 
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tive  symptoms  and  40.91  ±5.54  (mean  ±  SD) 
for  the  general  psychopathology  score. 

Apparatus 

The  apparatus  for  this  experiment  was  com¬ 
posed  of  a  personal  computer  (PC),  a  large 
screen,  a  beam  projector,  and  a  joystick,  which 
were  used  by  the  subjects  to  experience  a  VE. 
The  VE  consisted  of  a  room  and  a  maie-shaped 
virtual  avatar  that  had  eye  blink  and  breathing 
motions,  but  no  facial  expressions  and  no  ges¬ 
tures.  The  VE  and  avatar  were  rendered  on  a 
PC  using  “3D  Game  Studio”  (Conitec  Datasys- 
tems  Corporation),  which  is  a  commercial  3D 
game  engine,  and  projected  onto  a  big  screen 
through  a  beam  projector.  This  method  was 
chosen  because  a  patient  with  schizophrenia 
may  feel  discomfort  when  wearing  a  head 
mounted  display  (HMD)  and  tracker.  The  sub¬ 
jects  moved  forward  and  backward  by  pushing 
or  puiling  the  lever  on  the  joystick,  and  initiated 
conversation  by  pressing  the  talk  button  on  the 
joystick. 

Task  Description 

We  designed  the  task  so  that  the  participant 
had  to  initiate  and  conduct  a  short  conversation 
with  a  virtual  avatar.  Therefore,  a  subject  had  to 
approach  a  virtual  avatar  standing  in  front  of  the 
subject  by  pushing  the  lever  of  the  joystick  for¬ 
ward  and  try  to  talk  to  the  virtual  avatar  by 
pressing  the  talk  button  on  the  joystick.  In  order 
to  initiate  the  conversation,  subject  had  to  press 


the  “talk”  button  then  say  “hello”.  After  initiation, 
the  avatar  and  the  subject  exchanged  questions 
and  answers.  In  order  to  do  this,  we  generated 
a  fixed  simple  scenario  for  a  short  conversation 
and  the  clinician  controlled  the  scenario  accord¬ 
ing  to  the  subject’s  verbal  response  (for  a  de¬ 
tailed  script,  see  Fig.  2). 

Measurements 

For  VR-based  measurements,  we  required  the 
interpersonal  distance  and  the  verbal  response 
time  and  saved  them  during  the  task.  We  de¬ 
fined  former  as  a  virtual  distance  between  a 
subject  and  the  avatar  at  the  moment  that  the 
subject  approached  and  pressed  the  “talk”  but¬ 
ton  to  initiate  a  conversation.  It  is  reasonable  for 
the  distance  to  be  regarded  as  the  interpersonal 
distance,  since  the  distance  means  the  comfort¬ 
able  distance  for  having  a  conversation  with  the 
virtuai  avatar.  We  defined  the  verbal  response 
time  as  the  time  elapsed  following  the  moment 
that  the  avatar  asked  the  subject  a  question 
until  the  point  that  the  subject  commenced  the 
answer. 

The  PANSS,  used  to  measure  the  severity  of 
the  patients’  positive  and  negative  symptoms, 
allowed  assessment  of  the  relationship  between 
symptoms  and  behavioral  characteristics.  It  is 
composed  of  three  subscales:  the  negative  syn¬ 
drome  scale,  positive  syndrome  scale,  and  gen¬ 
eral  psychopathology  scale.  In  more  detail, 
each  subscale  is  determined  by  assessing  sev¬ 
eral  representative  schizophrenic  symptoms. 


Figure  1.  Scene  of  an  experiment  (a)  and  a  virtual  avatar  standing  in  a  virtual  room  (b).  A  subject  is 
sitting  on  a  chair,  a  screen  is  located  in  front  of  the  subject  and  the  subject  controls  a  joystick  to 
move  and  respond. 
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Script  for  a  short  conversation 


Subject :  Hello  ? 

Avatar  ;  Hello.  Nice  to  meet  you.  What  is  your  name 
Subject :  My  name  is  XXX.  What  is  your  name? 
Avatar :  My  name  is  000.  How  old  are  you? 
Subject :  I’m  28  years  old.  How  about  you? 

Avatar  :  I’m  29  years  old. 


he  verbal  response  time 


Figure  2.  Diagrammatic  representation  of  the  task  and  parameters,  and  a  script  for  a  short  conver¬ 
sation  for  this  task. 


For  example,  the  negative  syndrome  scale  has 

seven  characteristics  of  schizophrenia,  which 

are  blunted  affect,  emotional  withdrawal,  poor 

rapport,  passive  apathetic  social  withdrawal, 

difficulty  in  abstract  thinking,  lack  of  spontaneity 

&  flow  of  conversation  and  stereotyped  think- 
15 

mg. 

RESULTS 

The  mean  and  standard  deviation  of  the  inter¬ 
personal  distance  was  104.86  (SD=60.39)  in 
virtual  unit,  which  did  not  convert  to  a  real  unit, 
because  only  the  relative  difference  among  pa¬ 
tients  was  of  interest.  And,  when  we  investi¬ 
gated  the  relationship  between  the  distance  and 
PANSS  subscales,  the  distance  was  signifi¬ 
cantly  negatively  correlated  with  the  negative 
syndrome  score,  which  is  one  of  the  PANSS 
subscales  (n=11,  r=-0.687,  p=0.02),  but  we 
saw  no  correlations  with  other  subscales,  such 
as  positive  syndrome  and  general  psychopa¬ 
thology  score. 

The  mean  and  standard  deviation  of  the  verbal 
response  time  was  4.64  seconds  (SD=1.40). 
And,  when  we  investigated  the  correlation  be¬ 
tween  the  verbal  response  time  and  PANSS 
scores  and  its  subscales,  there  was  no  signifi¬ 
cant  correlation  with  positive  and  negative 


symptoms  and  the  general  psychopathology 
score. 

However,  there  were  two  significant  correla¬ 
tions;  namely,  between  verbal  response  time 
and  blunted  affect  characteristic  (n=11,  r=.638, 
p=.035),  and  between  verbal  response  time  and 
poor  rapport  (n=1 1 ,  r=.615,  p=.044). 

DISCUSSION 

This  study  investigated  social  response  be¬ 
tween  patients  with  schizophrenia  and  a  virtual 
avatar  so  that  we  could  find  the  patients’  behav¬ 
ioral  characteristics  to  a  virtual  avatar. 

The  negative  correlation  observed  in  this  study 
is  consistent  with  studies  that  investigated  the 
relationship  between  schizophrenic  patients’ 
interpersonal  distance  and  their  symptoms.  In 
particular,  our  results  of  a  negative  correlation 
are  consistent  with  a  study  conducted  by  Ne- 
chamkin  et  al.  showing  relationships  between 
the  distance  and  clinical  symptomatology,  as 
measured  by  the  PANSS  subscales  scores.^®  In 
that  study,  there  was  no  correlation  between 
positive  syndrome  scores  and  any  type  of  inter¬ 
personal  distances  measured  with  family, 
friends,  and  hostile  and  neutral  images,  while 
negative  syndrome  scores  exhibited  moderate 


146 


KU  ET  AL. 


Figure  3.  The  correlation  between  patients’  negative  syndrome  scale  and  the  interpersonal  distance. 


to  high  negative  correlations  with  the  distances 
measured  using  hostile  and  neutral  images. 
Explaining  our  results  based  on  the  Necham- 
kin’s  study,  patients  with  schizophrenia  per¬ 
ceived  the  avatar  as  a  neutral  or  hostile  human 
image.  It  is  reasonable  to  assume  that  the  pa¬ 
tients  with  schizophrenia  probably  perceived 
the  avatar  as  a  neutral  image  of  a  real  person 
because  the  avatar  did  not  show  any  facial  ex¬ 
pressions  and  gestures,  which  means  that  the 
avatar  showed  no  emotion  to  the  patients. 


The  additional  parameter  we  acquired  and  ana¬ 
lyzed  in  this  study  was  the  subject’s  verbal  re¬ 
sponse  time,  which  was  defined  as  the  time 
elapsed  between  the  patient's  response  and  the 
avatar’s  question.  Correlation  analysis  revealed 
positive  correlations  between  verbal  response 
time  and  both  the  blunted  affect  and  poor  rap¬ 
port  scales,  which  are  sub-items  of  the  PANSS 
negative  syndrome.  The  blunted  affect  score 
represents  the  amount  of  emotional  change, 
which  is  characterized  as  the  reduction  of  facial 


Subscales  in  the  negative  symptoms  of  PANSS 

Correlation  value  (p  value) 

N1:  Blunted  affect 

.638* 

.035 

N2:  Emotional  withdrawal 

.404 

.218 

N3:  Poor  rapport 

.615* 

.044 

N4:  Passive/apathetic  social  withdrawal 

.078 

.820 

N5:  Difficulty  in  abstract  thinking 

.400 

.222 

N6:  Lack  of  spontaneity  &  flow  of  conversation 

-.090 

.792 

N7:  Stereotyped  thinking 

-.109 

.749 

*  :  correlation  is  significant  at  the  0.05  level  (2-tailed) 

Table  1.  The  correlation  between  the  verbal  response  time  and  the  subscales  in  the  negative  symp¬ 
toms  of  PANSS. 


Annual  Review  of  CyberTherapy  and  Telemedicine 


expression,  emotional  modulation,  and  gesture 
during  communication.  By  contrast,  the  poor 
rapport  score  represents  the  amount  of  intimacy 
to  an  interviewer  or  the  level  of  deficit  in  forming 
a  relationship,  which  is  characterized  as  dis¬ 
tancing  a  human  relationship  and  the  reduction 
of  verbal  and  nonverbal  communication.^® 

The  positive  correlation  between  the  verbal  re¬ 
sponse  time  acquired  and  subscales  in  PANSS 
in  this  study  could  be  explained  by  the  definition 
of  these  subscales.  Therefore,  this  correlation 
means  that  the  less  a  patient’s  emotional  re¬ 
sponse,  intimacy,  and  relationship  making  with 
a  virtual  avatar,  the  slower  they  answer  the  ava¬ 
tar’s  question.  When  this  fact  is  considered  in 
view  of  a  virtual  avatar,  this  could  be  additional 
evidence  to  show  that  a  patient  with  schizo¬ 
phrenia  responds  to  a  virtual  avatar  in  the  same 
way  as  to  a  real  person. 

CONCLUSIONS 

Our  objective  for  this  study  was  to  investigate 
whether  VR  is  a  suitable  tool  for  extracting  and 
assessing  the  patient’s  behavioral  characteris¬ 
tics.  In  order  to  do  this,  we  investigated  the  be¬ 
havioral  and  verbal  response  characteristics  by 
observing  the  interpersonal  distance  between 
the  schizophrenic  patients  and  a  virtual  avatar, 
and  the  duration  from  the  time  the  avatar  asked 
a  question  to  the  time  the  subject  answered, 
respectively. 

This  study  provided  evidence  supporting  the 
position  that  the  patients  respond  to  the  virtual 
avatar  as  if  they  were  really  standing  near  it  and 
conversing.  It  is  also  supported  by  data  show¬ 
ing  that  the  patients’  behavioral  characteristics 
to  a  virtual  avatar  vary  in  accordance  with  their 
symptoms,  and  these  characteristics  are  the 
same  as  those  to  a  real  person.  This  indicates 
that  virtual  avatar  could  draw  patient’s  social 
response  in  an  objective  manner.  Therefore, 
this  study  could  be  meaningful  because  it  may 
provide  the  method  and  rationale  for  applying 
VR  techniques  to  training  or  assessing  the  be¬ 
havioral  and/or  cognitive  characteristics  of  pa¬ 
tients  with  schizophrenia. 
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Spatial  Ability  and  Navigation  Learning  in  a  Virtual  City 

J-H.  Lee,  Ph.D.^  ^  R.  Mraz,  BASc.,P.Eng.^  K.  K.  Zakzanis,  Ph.D.^ 

S.  E.  Black,  M.D.,Ph.D.\  P.  J.  Snyder,  Ph.D  ®,  S.  I.  Kim,  Ph.D.\  and  S.  J. 

Graham,  Ph.D.,  P.Eng.^'® 

^  Department  of  Biomedical  Engineering,  Hanyang  University,  Seoul,  Korea 
^  Imaging  Research,  Sunnybrook  &  Women's  College  Health  Science  Centre,  Toronto,  ON,  Canada 
®  Division  of  Life  Sciences, ''  Department  of  Medicine-Neurology,  and  ®  Department  of  Medical  Biophysics,  Uni¬ 
versity  of  Toronto,  ON,  Canada 
®Pfizer  Global  Research  &  Development,  Groton,  CT,  USA 

Abstract:  Background  &  Significance  of  the  Problem:  Virtual  environments  (VEs)  can  be  used  to  as¬ 
sess  spatial  memory  and  navigation  skiiis,  although  the  behaviour  elicited  is  inherently  complex.  The 
purpose  of  this  study  was  to  investigate  the  ecological  validity  and  interrelationship  between  virtual 
navigation  tasks  and  severai  neuropsychological  tests. 

Methods/Tools:  Forty-two  participants  (mean  age:  70.42)  took  part  in  the  study,  of  which  eighteen 
were  recruited  from  the  Cognitive  Neurology  Clinic  at  Sunnybrook  &  Women’s  Coliege  Health  Sci¬ 
ences  Centre  (S&W)  based  on  diagnosis  of  mild  cognitive  impairment  (MCI).  Subjects  completed  the 
Groton  Maze  Learning  Test  (GMLT;  a  hidden  maze  task  developed  by  one  of  the  authors  [P.J.S.]) 
and  several  neuropsychological  tests,  including  the  MMSE,^  the  Rey  Compiex  Figure  Test,^  Benton 
Visual  Retention  test,^  Trail-Making  Test  (Forms  A  and  B),^  and  Digit  Span.^  Sixteen  of  the  partici¬ 
pants  (9  normal,  7  MCI)  also  undertook  a  VE  navigation  task.^  Behaviour  elicited  by  the  VE  task  was 
characterized  in  terms  of  two  components,  as  estimated  by  factor  analysis:  a  VE  memory  index,  and 
a  VE  movement  index. 

Results:  The  VE  memory  index  tvas  significantly  associated  with  the  results  of  a  conventional  mem¬ 
ory  test  (Rey  Complex  Figure),  and  the  VE  movement  index  was  significantiy  associated  with  meas¬ 
ures  of  the  Trait  Making  Test.  Compared  with  normal  participants,  participants  with  mild  cognitive 
impairment  (MCI)  showed  a  significant  reduction  in  the  memory-related  measures  in  the  PGRD  Maze 
Learning  Test  and  memory  tests  (particuiarly  RCFT),  and  the  MCI  group  deviated  significantly  from 
the  correct  route,  particularly  in  novel  environments.  Performance  in  the  VE  navigation-learning  task 
was  significantly  associated  with  measures  of  visual  memory  and  the  executive  function  in  conven¬ 
tional  neuropsychological  tests. 

Conclusion:  The  results  of  this  study  demonstrate  that  VE  technology  can  be  used  to  assess  spatial 
memory  and  navigation  skills,  and  that  measures  of  this  behavior  in  a  VE  are  related  to  those  of  con¬ 
ventional  neuropsychological  tests.  Our  results  also  demonstrate  that  the  MCI  group  showed  a  sig¬ 
nificant  reduction  in  memory-related  measures  in  the  GMLT  and  the  neuropsychological  tests.  In  par¬ 
ticular,  differences  between  the  groups  were  evident  in  the  immediate  and  deiayed  recall  in  the  Rey 
Test.  However,  because  of  the  smail  sample  size,  statistically  significant  resuits  were  not  found  be¬ 
tween  groups  in  the  VE  navigation  task. 

Novelty:  The  interrelationship  between  the  VE  task,  the  GMLT,  and  neuropsychological  assessments 
is  important  for  administration  and  interpretation  of  these  computerized  tests  in  the  future. 
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Recently,  the  development  of  virtual  environ¬ 
ment  (VE)  technology  has  enabled  a  systematic 
and  laboratory-based  investigation  of  spatial 
navigation  in  humans.^  VE  technology  has  been 
used  successfully  to  assess  the  acquisition  of 
spatial  knowledge,^  sex  differences  in  spatial 
navigation,^  theta  oscillations  in  electroencepha- 
lography,"*  age  changes  in  route  learning,^  and 
regional  brain  activation  patterns  in  neuroimag¬ 
ing  studies.®  However,  VEs  have  limitations  in 
such  research:  typically,  participants  must  per¬ 
form  solely  on  the  basis  of  visual  information 
and  must  become  familiar  with  the  use  of  inter¬ 
faces  such  as  a  mouse  or  joystick  to  direct 
movements.  Nevertheless,  it  has  been  shown 
that  cognitive  maps  developed  using  VEs  are 
comparable  to  those  acquired  in  the  real  envi¬ 
ronment.^  Several  studies  have  confirmed  that 
representations  learned  in  VEs  are  transferable 
to  real  space.® 

Recent  efforts  to  study  human  spatial  memory 
and  navigation  have  involved  the  use  of  maps 
or  model  environments.  For  example.  Galea 
and  Kimura®  required  participants  to  learn  a 
route  through  a  town  drawn  on  a  large  sheet  of 
paper.  Although  the  elements  of  such  an  envi¬ 
ronment  can  be  well  controlled  by  the  experi¬ 
menter,  it  is  not  clear  that  learning  a  route  by 
perusing  a  2D  drawing  of  an  environment  uses 
the  same  spatial  skills  as  navigating  in  a  more 
ecologically  valid  environment.^®  Many  studies 
suggest  that  such  navigation  is  not  the  same  as 
a  table-top  test  of  spatial  memory,  and  that  di¬ 
rect  inferences  cannot  be  made  about  one  from 
the  other.  On  a  table-top,  all  information  is 
within  one  field  of  view  (i.e.,  allocentric  perspec¬ 
tive);  this  is  not  the  case  in  a  complex  environ¬ 
ment,  in  which  much  of  the  relevant  information 
is  unseen  at  any  specific  time  (i.e.,  egocentric 
perspective).^  Patients  with  topographical  mem¬ 
ory  impairment,  who  present  with  well  intact 
table-top  spatial  and  geographical  knowledge 
tests,  have  been  described  in  the  literature.” 
The  opposite  type  of  impairment  has  also  been 
observed.  Maguire  and  Cipolotti^^  reported  a 
patient  with  selective  preservation  of  navigation 
ability  in  the  context  of  profound  verbal  and  vis¬ 
ual  memory  impairment  and  poor  geographical 
knowledge,  confirming  the  double  dissociation 
between  navigation  and  table-top  spatial  tasks. 
There  have  been  few  studies  investigating  the 
ecological  validity  of  tests  of  spatial  skill.  Son- 


nenfeld^®  concluded  that  paper-and-pencil  spa¬ 
tial  performance  is  independent  of  true  way¬ 
finding  or  navigation  ability.  Kirasic”  found  no 
significant  relationship  between  elderly  individu¬ 
als’  performance  on  psychometric  tasks  and 
their  navigation  ability.  However,  she  found  that 
performance  on  a  photographic-slide  route- 
simulation  task  correlated  significantly  with  real- 
world  spatial  performance.  Recently,  Nadolne 
and  Stringer^®  reported  that  from  a  battery  of 
clinical  tests  administered  to  stroke  patients, 
one  test  of  visual  memory  correlated  with  an 
environment-specific  directional-orientation  cri¬ 
terion  task,  but  no  clinical  test  correlated  with 
route  navigation  ability.  In  addition,  perform¬ 
ance  on  a  task  based  on  ecological  simulation 
correlated  well  with  route  navigation  ability.  Col¬ 
lectively,  these  studies  first  suggest  that  tasks 
simulating  the  spatial  environment  realistically 
may  have  greater  ecological  validity  than  tradi¬ 
tional  psychometric  measures.  Second,  they 
indicate  the  need  for  further  validation  of  eco¬ 
logical  simulation  tasks. 

To  summarize,  computer-simulated  3D  VEs 
have  recently  been  shown  to  measure  human 
navigational  abilities  and  spatial  memory  suc¬ 
cessfully  in  reasonably  realistic  settings  that  are 
both  novel  and  well  controlled.  In  this  study,  we 
investigate  the  relationship  between  a  virtual 
navigation-learning  task,  several  neuropsy¬ 
chological  tests  that  measure  visual  memory 
abilities,  and  a  computerized,  table-top  maze¬ 
learning  test  that  measures  spatial  memory  and 
learning.  Participating  in  the  study  were  a  group 
of  healthy  elderly  adults,  as  prior  studies  of 
navigation  have  demonstrated  that  elderly 
adults  show  more  difficulty  than  younger  adults 
in  learning  and  retrieving  routes.^®  A  second 
group  included  patients  with  selective  memory 
complaints  and  diagnosed  with  mild  cognitive 
impairment  according  to  standard  criteria. 

METHODS 

Participants 

Forty-two  participants  (32  healthy  elderly,  18 
MCI  patients)  with  a  mean  age  of  70.42  (±7.75) 
years  (range  58-84  years)  were  recruited  from 
the  Cognitive  Neurology  Clinic  at  Sunnybrook  & 
Women’s  College  Health  Sciences  Centre 
(S&W)  in  Toronto.  The  two  groups  were 
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matched  with  respect  to  their  scores  on  the 
Mini-Mental  State  Examination  (MMSE),  years 
of  education,  and  age.  All  were  paid  US$50  for 
participating.  Only  a  subset  participated  in  the 
VE  task,  and  within  this  group  the  data  for  14 
participants  were  lost  as  a  result  of  a  computer 
malfunction  and  withdrawals  from  the  study  be¬ 
cause  of  nausea  or  dizziness  induced  by  VE 
exposure.  The  final  sample  size  for  the  naviga¬ 
tion  trials  in  the  VE  was  16  (9  healthy  elderly,  7 
MCI  patients). 

Neuropsychological  Tests 

Each  participant  performed:  the  Rey-Osterieth 
Complex  Figure  Test  (RCFT;  copy,  immediate 
recall,  delayed  recall);  the  Trail  Making  Test  (A 
and  B);  the  Digit  Span  subtest  (forwards  and 
backwards)  from  the  Wechsler  Adult  Intelligence 
Scale-Revised;  the  Dementia  Rating  Scale 
(total,  attention,  initiation,  construction,  conceptu¬ 
alization,  and  memory  components);  and  the 
Mini-Mental  Status  Examination. 

The  PGRD  Maze  Learning  Test 

The  PGRD  Maze  Learning  Test,^^  a  computer- 
based  neuropsychological  measure  of  immedi¬ 
ate-  and  short-term  retention  of  visuospatial  infor¬ 
mation,  is  based  on  the  original  work  of  Milner.^® 
The  goal  is  to  learn  over  a  series  of  trials  how  to 
navigate  from  table-top  perspective  through  a 
hidden  maze  and  capture  a  target.  Participants 
pressed  on  a  touch-screen  to  reveal  the  maze 
under  a  matrix  of  tiles.  There  were  untimed  and 
timed  components  for  either  a4x4ora10’<  10 
Chase  Test. 

VE  Navigation-learning  Task 

A  virtual  city  (adapted  from  Mraz  et  al.^®)  was 
created  in  which  the  user  could  navigate  using  a 
projection  screen,  a  6-DOF  head  tracker,  and  a 
Joystick.  Participants  were  initially  trained  to 
move  in  virtual  space  by  traversing  an  elemen¬ 
tary  maze  consisting  of  12  turns  over  a  fixed  dis¬ 
tance.  Subsequently,  participants  were  shown  a 
path  through  the  city  with  video  playback  in  ego¬ 
centric  perspective,  and  alternately  watched  the 
route  or  attempted  to  navigate  it  themselves. 
They  were  instructed  to  navigate  as  accurately 
as  possible  without  bumping  into  objects,  to  pay 
attention  to  “the  rules  of  the  road”  by  staying  on 


the  sidewalk,  and  to  pay  attention  to  where  vari¬ 
ous  objects  in  the  city  were  located  with  respect 
to  each  other.  A  set  of  three  learning  trials  was 
performed  for  one  path  (Path  A),  followed  by  one 
learning  trial  for  a  second  path  (Path  B).  Short 
delay  (5  min.)  and  long  delay  (20  min.)  recall  tri¬ 
als  were  also  performed,  during  which  Path  A 
was  navigated  from  memory. 

PROCEDURES 

Informed  consent  was  obtained  from  each  par¬ 
ticipant,  based  on  approval  of  the  S&W  Research 
Ethics  Board.  Data  collection  for  all  participants 
occurred  in  the  following  order:  (1)  the  CDR  and 
the  MMSE;  (2)  the  neuropsychological  tests;  (3) 
the  PGRD  Maze  Learning  Test  (Trials  1  to  10, 
delay,  and  reverse);  (4)  the  VE  navigation- 
learning  task  followed  by  completion  of  the  Simu¬ 
lator  Sickness  Questionnaire^®.  Statistical  analy¬ 
sis  was  performed  using  commercial  software 
(SPSS  11.0). 

RESULTS 

An  index  of  memory  was  estimated  from  the 
neuropsychological  tests  by  factor  analysis: 
memory  index=0.934xZ(Rey  delayed  recall) 
+0.904xZ(Rey  immediate  recall)+  0.61  IxZ 
(forward  digit  span)+0.530xZ(CDR  memory). 
Internal  consistency  of  the  memory  index  was  a= 
0.772.  This  memory  index  and  the  data  from 
learning  tests  were  compared  to  determine 
whether  performance  on  paper-and-pencil  tests 
was  similar  to  that  on  the  computerized  tests. 
Consistently  high  and  significant  correlations 
were  found  between  the  memory  index  and 
measures  of  the  PGRD  Maze  Learning  Test.  In 
contrast,  only  two  measures  of  the  VE  task  were 
significantly  correlated  with  the  memory  index: 
time  to  completion  on  Trial  1  and  Long  Delay 
Recall.  This  initial  finding  was  then  used  to  per¬ 
form  an  additional  factor  analysis  providing  an 
overall  VE  navigation  index,  consisting  of  a  mem¬ 
ory  component  and  a  movement  component. 
The  VE  movement  index  was  composed  of 
measures  in  only  Trial  1  (distance  traveled  and 
wrong  turn  metrics,  specifically  excluding  time) 
because  the  other  trials  involved  repeated  expo¬ 
sure: 

VE  memory  index  =  Z(long  delay  time)  +  Z(Trial 
1  time) 
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VE  movement 

VE  memory 

VE  movement 

VE  memory 

0.32 

Memory  index 

-0.13 

-0.55* 

Rey  immediate  recall 

-0.25 

-0.12 

-0.60** 

Rey  delayed  recall 

-0.21 

-0.05 

-0.54* 

Trail  Making  Test  A 

0.35 

0.46** 

0.08 

Trail  Making  Test  A  error 

0.49 

0.41* 

0.08 

Trail  Making  Test  B  error 

0.58* 

0.47** 

0.23 

p<0.05,  **  p<0.01 


Table  1.  Pearson  Correlations  between  the  VE  indexes  and  Neuropsychological  Tests 


VE  movement  index  =  0.870xZ(wrong  turn)  + 
0.734xZ(distance)  +  0.598xZ(double  wrong 
turn) 

VE  navigation  index  =  VR  memory  index  +  VR 
movement  index 

The  VE  indices  were  subsequently  analyzed  to 
confirm  the  validity  with  the  neuropsychological 
tests.  Table  1  illustrates  that  the  VE  navigation 
index  was  more  highly  correlated  with  the  VE 
movement  index  than  with  the  VE  memory  in¬ 
dex,  and  showed  significant  correlation  with  the 
Trail  Making  Test  (A  error,  and  B  error),  which 
measured  an  executive  function  and  visuomotor 
tracking.  The  VE  movement  and  memory  indi¬ 
ces  were  not  statistically  correlated  with  each 
other.  The  VE  movement  index  also  was  signifi¬ 
cantly  correlated  with  the  TMT  measures  (A,  A 
error,  B  error),  whereas  the  VE  memory  index 
was  correlated  with  measures  of  the  Rey  imme¬ 
diate  recall  and  delayed  recall. 

A  two-factor  repeated-measures  ANOVA  was 
used  to  analyze  the  measures  across  the  trials 


of  the  PGRD  test  and  the  VE  navigation¬ 
learning  task,  for  both  normal  and  MCI  groups. 
With  respect  to  the  PGRD  test,  a  significant 
main  effect  of  the  trials  was  revealed  for  com¬ 
pletion  time,  F(1,37)=84.595,  p<0.01,  indicating 
improved  performance  as  the  trials  progressed. 
There  was  no  effect  of  group  (normal  versus 
MCI)  on  overall  performance,  nor  was  there  an 
interaction  between  group  and  trial  number. 
Other  measures  (movement,  error,  and  per¬ 
severation)  of  the  PGRD  test  gave  the  same 
results.  On  some  specific  measures  that  were 
difficult  or  required  greater  memory  resources, 
however,  such  as  the  delay  and  reverse  tests, 
there  were  significant  differences  between 
healthy  elderly  and  MCI  patients.  These  find¬ 
ings  were  mirrored  in  the  neuropsychological 
tests  and  are  shown  in  Table  2. 

For  the  VE  navigation-learning  test,  a  significant 
main  effect  of  trials  for  completion  time  and  sin¬ 
gle  wrong  turns  was  found:  F(1,12)=12.725, 
p<0.01;  F(1,12)=7.824,  p<0.05,  respectively.  As 


Variables 

Normal 

MCI 

t 

PGRD 

maze 

learning 

test 

Trial2_rule  break 
Delay  time 

Reverse  time 

0.26±0.45 

41.05±14.36 

46.95±16.57 

0.83±1.04 

55.95±23.81 

65.98±25.51 

-2.372* 

-2.52* 

-2.82** 

Neuropsy¬ 

chological 

tests 

CDR  total 

CDR  initiation 

Rey  Immediate  Re¬ 
call 

Rey  Delayed  Recall 
Memory  index 

140.00±3.12 

36.67±0.64 

18.94±6.13 

18.58±5.53 

52.56±11.00 

137.28±4.82 

35.33±2.40 

11.33±6.39 

10.62±5.93 

37.77111.68 

2.22* 

2.61** 

3.90** 

4.41** 

4.14** 

*  p<0.05,  **  p<0.01 

Table  2.  Performance  of  Normal  and  MCI  Groups  on  Tests  (summarized  results) 
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above,  there  were  no  group  or  interaction  ef-  was  strongly  correlated  with  the  time  variables  of 

fects.  No  specific  measures  showed  significant  the  PGRD  Maze  Learning  Test  and  the  memory 

differences  between  the  groups  analogous  to  index  of  the  neuropsychological  tests  (from  Trials 
those  shown  in  Table  2.  2  to  9,  r=0.514~0.625).  These  results  suggest 

that  navigational-learning  in  a  VE  is  very  useful  in 
DISCUSSION  the  assessment  of  spatial  memory.  Although  not 

directly  assessed  in  this  study,  it  is  likely  that  the 
The  results  of  this  paper  demonstrate  that  VE  VE  navigational-learning  task  can  provide  reli- 

technology  can  be  used  to  assess  spatial  mem-  able  and  valid  measures  of  real-world  functional 

ory  and  navigation  skills,  and  that  the  measures  abilities.  Additional  research  is  required  to  ad- 

of  this  behavior  in  a  VE  are  related  to  those  of  dress  this  question, 

newer  and  conventional  neuropsychological 

tests.  The  VE  memory  index  was  related  to  the  Several  results  suggest  that  clinically  relevant 

RCFT  (immediate  and  delayed),  but  the  VE  impairments  in  navigation  skills,  spatial  orienta- 

movement  index  was  related  to  the  Trail  Making  tion,  and  memory  are  often  apparent  in  the  early 

Test,  which  is  known  to  assess  visuomotor  skills  stages  of  dementia  and  MCI.^^  Our  results  dem¬ 
and  executive  function.  The  VE  memory  index  onstrate  that  the  MCI  group  showed  a  significant 


(a)  Path  A  (Normal  group)  (b)  Path  A  (MCI  group) 

faWi  B  -  Wamiil  CaiHralt  , _ Path  B- MCI  Pall»nt» 
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reduction  in  a  subset  of  the  memory-related 
measures  in  the  PGRD  Maze  Learning  Test  and 
in  the  neuropsychological  tests.  In  particular, 
differences  between  the  groups  were  evident  in 
the  immediate  and  delayed  recall  in  the  RCFT. 
These  results  were  not  observed  for  the  VE  navi¬ 
gation-learning  task,  which  could  be  due  to  the 
reduced  sample  size  for  this  component  of  the 
study.  It  is  also  of  interest  to  visualize  the  navi¬ 
gational  differences  between  the  groups  when 
performing  the  VE  task  (Figure  1). 

Although  most  of  the  normal  group  (except  two 
participants)  did  not  deviate  from  the  correct 
path,  the  MCI  group  tended  to  deviate  from  the 
correct  path,  particularly  Path  B  (Figure  4(d)). 
These  results  suggest  that  the  MCI  group  en¬ 
countered  greater  difficulty  than  the  normal 
group  in  the  novel  environments.  They  also 
highlight  one  of  the  recognized  challenges  in 
the  interpretation  of  behavior  elicited  in  virtual 
environments.  Given  that  the  behavior  can  be 
complex,  it  is  likely  that  characterizing  behavior 
in  terms  of  simple  metrics  such  as  time  to  com¬ 
pletion  fails  to  encapsulate  the  richness  of  the 
data. 

Although  there  are  no  differences  in  the  effects 
of  sex  in  the  current  study,  studies  with  larger 
sample  sizes  across  the  groups  are  needed 
before  it  can  be  definitively  concluded  that  MCI 
groups  have  inferior  spatial  memory  and  navi¬ 
gation  skills  irrespective  of  age  and  sex.  Future 
studies  need  to  investigate  how  age,  sex,  and 
strategy  affect  spatial  memory  and  navigation 
learning  in  normal,  MCI,  and  Alzheimer  pa¬ 
tients. 

ACKNOWLEDGMENTS 

This  work  was  supported  through  grants  from 
Pfizer,  Inc.,  the  Canadian  Foundation  for  Inno¬ 
vation,  the  Ontario  Premier’s  Research  Excel¬ 
lence  Award,  and  the  Ontario  Research  and 
Development  Challenge  Fund.  The  authors 
thank  Le-Anh  Ngo  and  Mark  Boulos  for  their 
assistance  in  testing  the  participants. 

References 

1.  Maguire,  E.  A.,  Burgess,  N.,  &  O’Keefe,  J. 
(1999).  Human  spatial  navigation:  cognitive 
maps,  sexual  dimorphism,  and  neural  sub¬ 
strates.  Current  Opinion  in  Neurobiology 
9:171-177. 


2.  Gillner,  S.,  &  Mallot,  H.A.  (1998).  Navigation 
and  acquisition  of  spatial  knowledge  in  a 
virtual  maze.  Journai  of  Cognitive  Neurosci¬ 
ence  10:445-463. 

3.  Astur,  R.S.,  Ortiz,  M.L,  &  Sutherland,  R.J. 
(1998).  A  characterization  of  performance  by 
men  and  women  in  a  virtual  Morris  water 
task:  a  large  and  reliable  sex  difference.  Be¬ 
havioural  Brain  Research  93:185-190. 

4.  Kahana,  M.J.,  Sekuler,  R.,  Caplan,  J.B.,  Kir- 
schen,  M.,  &  Madsen,  J.R.  (1999).  Human 
theta  oscillations  exhibit  task  dependence 
during  virtual  maze  navigation.  Nature 
399:781-784. 

5.  Moffat,  S.D.,  Zonderman,  A.B.,  &  Resnick, 
S.M.  (2001).  Age  differences  in  spatial  mem¬ 
ory  in  a  virtual  environment  navigation  task. 
Neurobiology  of  Aging  22:787-796. 

6.  Gron,  G.,  Wunderlich,  A.P.,  Spitzer,  M., 
Tomczak,  R.,  &  Riepe,  M.W.  (2000).  Brain 
activation  during  human  navigation:  gender- 
different  neural  networks  as  substrate  of  per¬ 
formance.  Nature  Neuroscience  3:404-^08. 

7.  Ekstrom,  A.D.,  Kahana,  M.J.,  Caplan,  J.B., 
Fields,  T.A.,  Isham,  E.A.,  Newman,  E.L.,  & 
Fried,  I.  (2003).  Cellular  networks  underlying 
human  spatial  navigation.  Nature  425 
(6954):  184-1 88. 

8.  Regian,  J.W.,  &  Yadrick,  R.M.  (1994).  As¬ 
sessment  of  configurational  knowledge  of 
naturally-  and  artificially-acquired  large-scale 
space.  Journal  of  Environmental  Psychology 
14:211-223. 

9.  Galea,  L.A.M.,  &  Kimura,  D.  (1992).  Sex  dif¬ 
ferences  in  route-learning.  Personality  and 
Individual  Differences  14:53-65. 

10.  Sandstrom,  N.J.,  Kaufman,  J.,  &  Huettel, 
S.A.  (1998).  Males  and  females  use  different 
distal  cues  in  a  virtual  environment  naviga¬ 
tion  task.  Cognitive  Brain  Research  6:351- 
360. 

11.  McCarthy,  R.A.,  Evans,  J.J.,  &  Hodges,  J.R. 
(1996).  Topographic  amnesia:  spatial  mem¬ 
ory  disorder,  perceptual  dysfunction,  or  cate¬ 
gory  specific  semantic  memory  impairment? 
Journal  of  Neurology  Neurosurgery  and  Psy¬ 
chiatry  60:31 8-325. 

12.  Maguire,  E.A.,  &  Cipolotti,  L.  (1998).  The 
selective  sparing  of  topographical  memory. 
Journal  of  Neurology  Neurosurgery  and  Psy- 


156 


Lee  et  al. 


chiatry  65:903-909.  2075  Bayview  Ave. 

13.  Sonnenfeld,  J.  (1985).  Tests  of  spatial  skill:  1°''°!^^°’. 

A  validation  problem.  Man-Environment  Sys-  Email.  Simon. graham@sw.ca 

terns  15:107-120. 

14.  Kirasic,  K.C.  (1988).  Aging  and  spatial  cog¬ 
nition:  current  status  and  new  directions  for 
experimental  researchers  and  cognitive  neu¬ 
ropsychologists.  In  J.M.  Williams  &  C.J. 

Long  (eds.),  Cognitive  Approaches  to  Neuro- 
psychoiogy.  New  York:  Plenum. 

15.  Nadolne,  M.J.,  &  Stringer,  A.Y.  (2001).  Eco- 
logic  validity  in  neuropsychological  assess¬ 
ment:  prediction  of  wayfinding.  Journal  of  the 
International  Neuropsychological  Society 
7:675-682. 

16.  Kirasic,  K.C.  (1991).  Spatial  cognition  and 
behavior  in  young  and  elderly  adults:  impli¬ 
cations  for  learning  new  environments.  Psy¬ 
chological  Aging  6(1 ):  1 0-1 8. 

17.  Peter  J.  Snyder  (2001).  PGRD  Maze  Learn¬ 
ing  Test.  Pfizer  Inc. 

18.  Milner,  B.  (1965).  Visually-guided  maze 
learning  in  man:  Effects  of  bilateral  hippo¬ 
campal,  bilateral  frontal,  and  unilateral  cere¬ 
bral  lesions.  Neuropsychologia,  3,  317-338. 

19.  Mraz,  R.,  Hong,  J.,  Quintin,  G.,  Staines, 

W.R.,  Mcliroy,  W.E.,  Zakzanis,  K.K.,  Gra¬ 
ham,  S.J.  (2003).  A  platform  for  combining 
virtual  reality  experiments  with  functional 
magnetic  resonance  imaging.  CyberPsychol- 
ogy  &  Behavior  6{4):359-368. 

20.  Kennedy,  R.,  Lane,  N.,  Berbaum,  N.,  & 

Lilienthal,  M.  (1993).  A  Simulator  Sickness 
Questionnaire  (SSQ):  A  New  Method  for 
Quantifying  Simulator  Sickness.  Interna¬ 
tional  Journal  of  Aviation  Psychology  3:203- 
220. 

21.  Passini,  R.,  Rainville,  C.,  Marchand,  N.,  & 

Joanette,  Y.  (1995).  Wayfinding  in  dementia 
of  the  Alzheimer  type:  planning  abilities. 

Journal  of  Clinical  and  Experimental  Neuro¬ 
psychology  17(6): 820-832. 


Contact 

Simon  J.  Graham,  Ph.D.,  P.Eng. 
S650-lmaging  Research 
Sunnybrook  &  Women’s  College  Health  Sci¬ 
ences  Center 


157 


Annual  Review  of  CyberTherapy  and  Telemedicine 


158 


Annual  Review  of  CyberTherapy  and  Telemedicine 


Telepsychiatry: 

Psychiatric  Consultation  through  Videoconference  Clinical  Results 

C.  De  las  Cuevas  PhD,  MD 

Department  of  Psychiatry.  University  of  La  Laguna 
Canary  Islands  Health  Service 

Abstract:  Main  Outcome  Measures:  Efficacy  variables  included  scores  on  the  Clinical  Global  Im- 
pressions-Severity  of  Illness  (CGI-S)  and  -Improvement  (CGI-I)  scales  as  well  as  Global  Indexes 
(GSI,  PSD  I  and  PST)  from  SCL-90R.  Response  was  defined  as  a  CGI-I  score  of  1  or  2.  Reliable 
Change  Indexes  were  computed  in  SCL-90R  Global  Indexes  scores.  A  multivariate  logistic  regres¬ 
sion  analysis  was  carried  out  to  evaluate  the  effect  of  each  factor  associated  with  the  success  of 
telepsychiatry. 

Results;  With  respect  to  baseline  severity  of  illness,  the  overall  average  Clinical  Global  Impression 
(CGI-S)  score  was  between  4  (moderately  ill)  and  5  (markedly  ill).  Only  4  patients  dropped-out  pre¬ 
maturely  the  study.  The  patients'  mean  SCL-90-R  Global  Indexes  scores  decreased  over  time,  indi¬ 
cating  less  psychiatric  distress.  Significant  decreases  from  baseline  to  week  24  were  obtained  in  psy¬ 
chiatric  status.  A  significant  proportion  of  patients  showed  clear  clinical  state  improvement.  Multivari¬ 
ate  logistic  regression  analysis  shows  that  tele  psychiatry  was  effective  independently  of  gender,  age, 
educational  level,  ICD-10  diagnosis,  initial  severity  of  illness  or  the  presence  of  antecedents  of  previ¬ 
ous  psychiatric  disorders  and  treatments. 

Conclusion:  Telepsychiatry  showed  to  be  an  effective  mean  of  delivering  mental  health  services  to 
psychiatric  outpatients  living  in  remote  areas  with  limited  resources. 


INTRODUCTION 

Telepsychiatry  can  be  conceived  as  an  inte¬ 
grated  system  of  mental  health  care  delivery 
that  employs  telecommunications  and  comput¬ 
erized  information  technology  as  an  alternative 
to  face-to-face  conventional  modality.  Video- 
conferencing  is  the  central  technology  currently 
used  in  telepsychiatry,  since  it  permit  live,  two- 
way  interactive,  full-colour,  video,  audio  and 
data  communication.  Telepsychiatry,  in  the 
form  of  videoconferencing,  has  been  well  re¬ 
ceived  in  terms  of  increasing  access  to  care 
and  user  satisfaction.^'^  Questions  persist,  how¬ 
ever,  about  its  effectiveness  since  there  are  few 
clinical  outcome  studies  and  because  there 
may  also  be  a  positive  reporting  bias  in  the  lit¬ 
erature.^ 

The  Canary  Islands  Health  Service  developed 
in  2003  a  telepsychiatry  programme  to  comple¬ 
ment  the  mental  health-care  of  the  citizens  liv¬ 
ing  on  La  Gomera  island  with  the  objectives  of 
improving  access,  reducing  isolation  and  im¬ 
proving  the  quality  of  mental  health  care  in  this 


area.  This  new  system  of  service  delivery  was 
possible  thanks  to  a  research  grant  (EU  QLRT- 
2001-0167  -  ISLANDS  Project)  from  the  Euro¬ 
pean  Union.  In  order  to  investigate  the  effec¬ 
tiveness  of  a  routine  telepsychiatry  service 
through  videoconferencing  a  prospective  cohort 
study  of  a  sample  of  70  psychiatric  outpatients 
was  carried  out. 

MATERIAL  AND  METHODS 

The  telepsychiatry  service  provides  psychiatric 
consultations  to  individuals  after  referral  from  a 
general  practitioner.  A  patient  living  in  La  Gom¬ 
era  with  a  mental  health  problem  can  choose 
between  joining  the  waiting  list  to  see  the  visit¬ 
ing  psychiatrist,  that  travel  every  Monday  from 
Tenerife  island,  or  being  included  in  the  telep¬ 
sychiatry  programme.  Telepsychiatry  sessions 
take  place  every  Thursday  from  9:00  to  13:00. 
Emergency  access  is  available  from  Monday  to 
Friday  (8:00  to  15:00).  After  the  teleconsulta¬ 
tion,  recommendations  are  provided  directly  to 
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Table  1,  Sociodemographic  and  clinical  characteristics  of  the  sample  studied 


Variable 

Category 

Number 
of  cases 

%of 

sample 

%  24  week 
CGI-Sev  <  2 

%  24  week 

CGI-  Imp  <  2 

Age 

<  25  years 

12 

17.1 

91.7 

100 

25-45  years 

37 

52.9 

67.6 

78.3 

45-65  years 

16 

22.9 

50 

68.7 

>  65  years 

5 

7.1 

60 

80 

Gender 

Male 

22 

31.4 

72.7 

86.4 

Female 

48 

68.6 

64.6 

77  A 

Educational  level 

Can  read  and  write 

11 

15.8 

63.6 

81.8 

Primary  studies 

33 

47.1 

95.6 

78.8 

College 

13 

18.6 

46.1 

69.2 

University  degree 

13 

18.6 

92.3 

92.3 

ICD-10  Diagnosis  * 

FI 

5 

7.1 

80 

80 

F2 

5 

7.1 

80 

80 

F3 

23 

32.9 

47.8 

69.6 

F4 

31 

44.3 

77.4 

83.9 

F6 

6 

8.6 

66.6 

100 

CGI  -  Severity  of  illness 

Moderatelly  ill 

8 

11.4 

62.5 

62.5 

Markedly  ill 

61 

87.1 

68.8 

83.6 

Severely  ill 

1 

1.4 

0 

0 

Total 

70 

100 

67,2% 

80% 

*  iCD-10  Diagnoses 

(F1):  Mental  and  behavioural  disorders  due  to  psychoactive  substance  abuse 

(F2):  Schizophrenia,  schizotypal  and  delusional  disorders 

(F3);  Mood  (affective)  disorders 

(F4):  Neurotic,  stress-related  and  somatoform  disorders 

(F6):  Disorders  of  the  adult  personality  and  behaviour 

%  24  week  CGI  -  Sev  <  2:  Proportion  of  patients  with  CGI  Severity  of  Illness  score  <  2  (1=  normal, 
not  at  all  ill;  2=borderline,  mentally  ill)  at  week  24. 

%  24  week  CGI  -  Imp  <  2:  Proportion  of  patients  with  CGI  Global  Improvement  score  <  2  (1=  very 
much  improved;  2=  much  improved)  at  week  24. 
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the  patient’s  GP  via  email.  Telepsychiatry  con¬ 
sultations  use  commercial  videoconferencing 
equipment  (Viewstation  512,  Polycom®)  con¬ 
nected  via  ISDN  lines  at  up  to  512  kbit/s. 

PATIENTS 

The  sample  comprised  70  psychiatric  outpa¬ 
tients  patients,  with  66.3%  female  and  33.7% 
male,  which  were  followed  up  through  24 
weeks.  The  mean  age  was  39.8±15  years 
(range  15-80).  Women  registered  a  higher 
mean  age  than  men  (42.9±15  vs.  33.2±12).  Pa¬ 
tients  were  diagnosed  according  to  ICD-10  cri¬ 
teria.  Anxiety  disorders  were  the  more  prevalent 
diagnosis.  Most  of  the  patients  (87.1%)  in¬ 
cluded  were  markedly  ill  at  the  beginning  and 
the  majority  of  them  (67.1%)  have  antecedents 
of  previous  psychiatric  treatments  in  face-to- 
face  alternative.  Of  70  patients  included  in  the 
study,  only  4  dropped-out  prematurely.  Other 
socio-demographic  and  clinical  characteristics 
of  the  sample  studied  are  shown  in  Table  1. 

TREATMENT 

The  telepsychiatry  treatment  was  conducted  by 
videoconference  between  the  University  Hospi¬ 
tal  de  la  Candelaria  in  Santa  Cruz  de  Tenerife 
(psychiatrist  location)  and  the  Mental  Health 
Care  Health  Centre  of  San  Sebastian  de  la 
Gomera.  (Patients  location).  Treatment  involves 
at  least  6  sessions  lasting  30  minutes  over  the 
24  week  study  period.  Additional  treatment  ses¬ 
sions  take  place  if  clinically  indicated.  The  treat¬ 
ment  consists  of  pertinent  psychotropic  medica¬ 
tion  plus  cognitive-behavioural  treatment  and 
psychological  evaluation  concerning  the  dis¬ 
ease,  medications,  and  side  effects.  The  mean 
number  of  psychoactive  drugs  prescribed  was 
1.76±0.84  (range  0-4).  Antidepressants  were 
the  most  prescribed  drugs  (74.3%),  followed  by 
benzodiazepine  tranquillisers  (72.9%)  and  an- 
tipsychotics  (14.3%). 

INSTRUMENTS 

The  efficacy  of  treatments  received  was  meas¬ 
ured  by  changes  in  the  Symptom  Checklist-90 
Revised  (SCL-90R)''  global  distress  indices  and 
Clinical  Global  Impression  (CGI)  (NIMH,  1970) 
ratings  at  weeks  0,  4,  8,  12,  16,  20  and  24. 


The  Symptom  Checklist-90  Revised  is  a  stan¬ 
dardized  multidimensional  90-item  self-report 
symptom  inventory  covering  various  dimen¬ 
sions  of  psychological  distress.  Each  item  is 
rated  on  a  5-point  scale  of  distress,  ranging 
from  not  at  all  (0)  to  extremely  (4).  It  utilizes 
three  global  distress  indexes;  Global  Severity 
Index  (GSI),  Positive  Symptom  Distress  Index 
(PSDI)  and  Positive  Symptom  Total  (PST).  The 
GSI  combines  the  number  of  symptoms  re¬ 
ported  and  intensity  of  reported  distress  to  yield 
the  single  best  descriptor  of  current  mental 
health.  The  PSDI  represents  the  average  level 
of  distress  reported  for  symptoms  endorsed 
and  the  PST  reflects  the  total  number  of  symp¬ 
toms  reported  regardless  of  symptom  intensity. 
Normative  data  for  Canary  Islands  citizens 
were  available  and  were  used  to  calculate  reli¬ 
able  changes  indexes. 

Clinical  Global  Impression  is  a  three-item  scale 
used  to  assess  treatment  response  in  all  cate¬ 
gories  of  psychiatric  patients.  It  was  adminis¬ 
tered  by  patient's  psychiatrist  and  2  minutes  are 
enough  to  complete.  The  items  are;  Severity  of 
Illness;  Global  Improvement  and  Efficacy  Index. 
Item  1  is  rated  on  a  seven-point  scale 
(1=normal  to  7=extremely  ill);  item  2  on  a 
seven-point  scale  (1=very  much  Improved  to 
7=very  much  worse);  and  item  3  on  a  four-point 
scale  (from  'none'  to  'outweighs  therapeutic 
effect'). 

PROCEDURE 

In  the  same  way  that  Jacobson  and  Truax,®  we 
defined  “reliable  change”  in  terms  of  the  reliabil¬ 
ity  of  the  measurement  instruments  used.  We 
considered  that  the  error  variance  in  a  set  of 
scores  that  is  due  to  the  unreliability  of  the 
scale  is  the  standard  error  of  measurement. 
Scales  that  are  highly  reliable  will  have  a  small 
standard  error  of  measurement.  If  we  know  the 
reliability  of  the  scale  (typically  measured  as 
Cronbach's  alpha)  and  the  standard  deviation 
of  the  raw  scores  on  that  scale  we  can  find  the 
expected  standard  deviation  of  the  variability  of 
the  error  scores.  The  formula  for  the  standard 
error  measurement  is; 


SSmeoi  =  Owittj  =  SD  *  ~Jl  —  ni 


Where  SD  =  the  standard  deviation  of  the 
measure,  and  rii=  the  reliability  (typically 
coefficient  alpha)  of  the  measure. 
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A  Reliable  Change  Index  (RCI)  is  computed  by 
dividing  the  difference  between  the  pretreat¬ 
ment  and  posttreatment  scores  by  the  standard 
error  of  the  difference  between  the  two  scores. 
If  the  RCI  is  greater  than  1 .96,  then  the  differ¬ 
ence  is  reliable,  a  change  of  that  magnitude 
would  not  be  expected  due  to  the  unreliability  of 
the  measure.  Conversely,  if  the  RCI  score  is 
1 .96  or  less  then  the  change  is  not  considered 
to  be  reliable,  it  could  have  occurred  just  due  to 
the  unreliability  of  the  measure. 


RCI  =  (posttest  -  pretest)  /  SEmeas 


In  order  to  evaluate  the  effect  of  each  factor 
associated  with  the  success  of  telepsychiatry,  a 
multivariate  logistic  regression  analysis  was 
carried  out. 

RESULTS 

Clinical  changes  in  CGI  Severity  and  Improve¬ 
ment  scores  after  treatment  via  telepsychiatry 
are  provided  in  Table  1.  This  shows  that  telep¬ 
sychiatry  was  effective  in  significantly  reducing 
clinical  measures  of  severity  as  determined  by 
a  CGI  change  score  of  2  or  less  (normal,  not  at 
all  ill  or  borderline,  mentally  ill)  in  67.2%  of  the 
sample  at  week  24.  In  the  same  way,  80%  of 
the  patients  included  in  the  study  were  "much" 
or  "very  much"  improved  on  the  clinical  global 
impression  scale.  Figure  1  shows  the  mean 
CGI  Severity  of  Illness  scores  at  baseline,  week 
8,  week  16  and  week  24. 


Figure  1.  Mean  CGI  Severity  of  Illness  scores 
at  baseline,  week  8,  week  16  and  week  24 


4.9 


Baseline  Week  8  Week  16  Week  24 


■  Mean  CGI  Severity 


The  patients'  mean  SCL-90-R  Global  Indexes 
scores  decreased  over  time,  indicating  less 
psychiatric  distress.  Statistical  summaries  of 
the  data,  including  reliable  change  index  data, 
are  displayed  in  Table  2.  Only  3%  of  patients 
that  finished  the  24  weeks  follow  up  reported  a 
reliable  deterioration  in  their  mental  health 
status  according  GSl,  while  22.7%  reported 
uncertain  changes  at  the  end  of  the  videocon¬ 
ferencing  treatment  period.  74.3%  of  patients 
reported  a  reliable  improvement  in  their  CGI 
scores,  being  45.5%  recovered.  Concerning  the 
level  of  distress  informed,  68.1%  of  the  patients 
reported  reliable  improvements,  being  62.1% 
recovered.  Considering  the  total  number  of 
symptoms  reported,  52.1%  informed  reliable 
improvement,  being  only  39.6%  recovered.  It  is 


Table  2.  Reliable  Change  Index  Summary  Statistics  of  SCL-90R  Global  Indexes 

(n  =  66) 

Reliable  dete¬ 
rioration 

Uncertain 

change 

Reliable  im¬ 
provement  - 
not  recovered 

Reiiable  im¬ 
provement- 
recovered 

%  Moved  from 
above  cutoff 
score  at  pretest 
to  below  cutoff 
score  at  follow¬ 
up 

Measure 

n 

% 

n 

% 

n 

% 

n 

% 

n 

% 

SCL-90R 

GSl 

2  ' 

3 

15 

22,7 

19 

28,8 

30 

45,5 

35  of 
66 

53 

SCL-90R 

PSDI 

2 

3 

19 

28,8 

D 

6 

41 

62,1 

54  of 
66 

82 

SCL-90R 

PST 

1 

1-5 

30 

45,4 

11 

13,5 

24 

39,6 

30  of 
66 

45,5 
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Variable 

Statistic  value  of 
Wald 

Degrees  of  Free¬ 
dom 

P  value  of  Wald 

Gender 

2,454 

1 

,117 

Age 

2,296 

1 

,130 

Educational  Level 

3,741 

5 

,587 

ICD-10  Diagnoses 

,428 

4 

,980 

Antecedents 

,041 

1 

,839 

Severity  of  Illness 

,299 

2 

,861 

Table  3.  Multivariate  logistic  regression  analysis  results.  Variables  in  the  equation 


Pretreatment 


Figure  2.  Graphic  Representation  of  Reliable 
Change  Index  in  Global  Severity  Index 

Reliable  deterioration  are  those  cases  in  the  up¬ 
per  right  triangle,  outside  of  the  band  of  no  reli¬ 
able  change.  Uncertain  change  are  those  par¬ 
ticipants  within  the  band  of  no  reliable 
change.  Reliable  improvement  -  not  recovered 
are  individuals  to  the  right  of  the  band  of  no  reli¬ 
able  change  and  above  the  1  SD  cutoff 
score.  Reliable  improvement  -  recovered  are 
individuals  to  the  right  of  the  band  of  no  reliable 
change  and  below  the  1  SD  cutoff  score. 


necessary  to  consider  that  almost  all  patients 
received  drug  treatments  that  involve  adverse 
effects  that  bias  this  figures. 

None  of  the  variables  included  in  the  study  in¬ 
fluenced  in  a  significant  way  the  success  of  the 
treatment  (Table  3).  Neither  the  gender,  nor  the 
age  or  the  educational  level  of  the  patients 
showed  statistically  significant  differences  in  the 
rate  of  telepsychiatry  success.  In  the  same 
way,  neither  the  diagnosis  of  the  patient,  nor 
the  existence  of  antecedents  of  previous  psy¬ 
chiatric  disorders  treated,  or  the  severity  of  the 
illness  the  beginning  of  treatment  influenced  the 
success  of  the  telepsychiatry  treatment. 

As  conclusion  of  the  exposed  results,  telepsy¬ 
chiatry  showed  to  be  an  effective  mean  of  deliv¬ 
ering  mental  health  services  to  psychiatric  out¬ 
patients  living  in  remote  areas  with  limited  re¬ 
sources.  Its  clinical  efficacy  was  not  conditioned 
by  any  of  the  analysed  variables. 
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Figure  3.  Graphic  Representation  of  Reliabie 
Change  Index  in  Positive  Symptom  Distress 
Index 
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Training  Blind  Children  to  Develop  Mathematics  Skills  Through  Audio 
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Abstract:  A  number  of  studies  using  computer  applications  have  been  designed  to  improve  learning 
in  children  with  visual  disabilities.  Some  research  studies  use  audio-based  interactive  interfaces  to 
enhance  learning  and  cognition  in  children  with  visual  disabilities.  The  development  of  mathematics 
learning  through  virtual  environments  has  not  been  studied  in  the  literature.  This  work  presents  the 
design,  development,  and  usability  of  two  audio-based  virtual  environments:  AudioMath  and  Theo  & 
Seth,  interactive  virtual  environments  based  on  audio  to  develop  mathematics  learning  of  children 
with  visual  disabilities.  AudioMath  and  Theo  &  Seth  were  developed  by  and  for  blind  children.  They 
participated  in  the  design  and  usability  tested  the  software  during  and  after  implementation.  Our  re¬ 
sults  evidenced  that  audio  embedded  into  virtual  environments  can  be  a  powerful  interface  to  de¬ 
velop  mathematics  learning  in  blind  children. 


INTRODUCTION 

Virtual  environments  based  on  audio  to  foster 
learning  and  cognition  has  been  increasingly 
developed  for  people  with  visual  disabilities. 
Diverse  studies  have  focused  on  the  design  of 
audio-based  interfaces  and  evaluated  their  im¬ 
pact  on  learning  and  cognition.^'®  Most  of  them 
are  based  on  interactive  software  that  cannot 
be  fully  adapted  to  their  needs  and  require¬ 
ments.  Actually,  some  of  them  are  fixed  proto¬ 
types  without  enough  flexibility  and  versatility. 

A  seminal  study  developed  the  first  proof-of- 
concept  application  to  navigate  virtual  environ¬ 
ments  through  audio  by  blind  children.^  The 
study  used  audio  to  enhance  spatial  cognitive 
structures  and  found  that  spatial  sound  can  be 
used  to  develop  spatial  navigation  skills  in  a 
virtual  environment.  Some  studies  designed 
experiences  with  audio  stimuli  to  simulate  vis¬ 
ual  cues  for  blind  learners, finding  that  by  us¬ 
ing  3D  audio  interfaces  blind  people  can  help  to 
localize  a  specific  point  in  a  3D  space.  They 
performed  with  precision  but  slower  than 
sighted  people  concluding  that  navigating  vir¬ 
tual  environments  with  only  sound  can  be  more 
precise  to  blind  people  in  comparison  to  sighted 
persons.'* 

Another  study  describes  the  positive  effect  of 
3D  audio-based  virtual  environments.®  A  study 
in  the  same  line  of  research  used  sensory  vir¬ 


tual  environments  through  force  feedback  joy¬ 
sticks  simulating  real  places  such  as  the  school 
or  work  place.  They  probed  the  hypothesis  that 
by  providing  appropriate  spatial  information 
through  compensatory  channels  the  perform¬ 
ance  of  blind  people  can  be  improved.*® 

A  research  project  in  the  same  direction  of^ 
concluded  that  a  traditional  computer  game, 
Space  Invader,  can  be  replicated  with  3D 
sound.  Researchers  used  force  feedback  joy¬ 
sticks  as  input  interface  by  letting  to  play  blind 
to  sighted  children  to  share  the  same  experi¬ 
ence.®  An  interesting  study  tested  blind  and 
sighted  people  with  covered  eyes  across  audio 
stimuli  by  tracing  specific  places  through  sound. 
The  skill  to  hold  in  mind  the  specific  localization 
without  concurrent  perceptual  information  or 
spatial  update  was  evaluated.** 

A  trend  in  the  literature  is  the  absence  of  long¬ 
term  usability  studies  on  audio-based  virtual 
environments.  The  literature  sustains  that  spa¬ 
tial  audio  may  have  a  reduced  impact  when  it  is 
not  associated  to  specific  cognitive  tasks.  There 
is  also  a  demand  for  more  rigorous  and  system¬ 
atic  usability  studies  by  and  for  children  with 
visual  disabilities. 
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Learning  the  basics  of  mathematics  has  been  a 
current  issue  in  literacy  literature.  Most  studies 
worldwide  agree  that  children  do  not  learn 
mathematics  adequately  in  the  early  grades. 
This  has  a  tremendous  impact  on  further  learn¬ 
ing.  In  a  world  heavily  based  on  science  and 
technology  children  without  understanding  the 
basics  of  math  limit  their  role  in  the  society. 
Children  with  visual  disabilities  are  not  the  ex¬ 
ception.  Actually  in  many  respects  this  issue  is 
radicalized  in  these  children.  When  blindness  is 
associated  to  social  deprivation  the  issue  of 
learning  primary  school  mathematics  is  really  a 
critical  issue.^^'^"'  Thus  one  of  the  greatest  chal¬ 
lenges  for  children  with  visual  disabilities  has 
been  the  access  and  learning  of  mathematics 
information.^^  Early  learning  and  practice  may 
project  a  better  construction  of  mathematics 
knowledge  in  visually  impaired  children. 

In  this  study  we  intended  to  foster  learning  and 
practice  of  mathematical  concepts  such  as  po¬ 
sitional  value,  sequences,  additive  decomposi¬ 
tion,  addition,  sum,  subtraction,  multiplication, 
and  division.  Theo  &  Seth  was  developed  to 
enhance  mathematics  learning  such  as  the 
concept  of  number  and  basic  operations. 
AudioMath  is  used  to  assist  learners  with  visual 
disabilities  in  learning  concepts  such  as  estab¬ 
lishing  correspondence  and  equivalency  rela¬ 
tionships,  memory  development,  and  differenti¬ 
ating  tempo-spatial  notions. 

DESIGNS 

Models.  AudioMath  has  different  components. 
Specific  content  models  the  representation 
problem  to  generate  a  grid  with  a  pair  of  related 
tokens  between  them  to  be  solved  by  the  child. 
Random  card  generator  is  the  editor  that  allows 
setting  the  level  of  complexity  and  contents 
from  a  gallery.  Computer  model  is  the  repre¬ 
sentation  of  the  real  problem.  It  includes  state 
system  variables  such  as  number  of  correct 
token  pairs,  time,  and  score  as  well  as  para¬ 
metric  variables  such  as  level,  content,  and 
user  name.  Projection  implies  transforming  in¬ 
put  signals  to  changes  perceived  by  blind  users 
either  audibie  or  tactile.  It  bridges  system  and 
interfaces  through  bidirectional  feedback  from 
and  toward  the  user  actions.  Interface  includes 
input/output  interaction  devices  such  as  audio, 
keyboard,  mouse,  force  feedback  joystick,  and 
tablets. 


The  model  is  based  on  a  matrix  with  rows  and 
columns.  There  are  four  levels  of  complexity: 
Level  1  with  four  tokens  (two  rows  and  two  col¬ 
umns),  level  2  with  six  tokens  (three  rows  and 
two  columns),  level  3  with  twelve  tokens  (three 
rows  and  four  columns),  and  level  4  with  six¬ 
teen  tokens  (four  rows  and  four  columns).  Col¬ 
ors  are  used  for  children  with  residuai  vision.^® 
This  model  meets  the  minimum  standards  pro¬ 
posed  by^®  for  software  design,  and  thus  vali¬ 
dating  AudioMath  as  an  appropriate  virtual  en¬ 
vironment  to  be  used  for  the  learning  of  children 
with  visual  disabilities. 

The  model  of  Theo  and  Seth  has  two  compo¬ 
nents  that  define  the  interaction  of  the  learner 
with  the  application.  Virtual  Environment  The 
composing  elements  are:  Interface  Controls  (in 
charge  of  controlling  the  volume),  help  (a  sys¬ 
tem  to  orient  the  user  to  use  the  keystrokes  in 
the  application),  and  activities  (containing  the 
specific  contents  of  mathematics).  Interface: 
The  elements  of  I/O  interfaces  are  the  keyboard 
and  the  audio  system.  The  interface  coordi¬ 
nates  the  user  interaction  with  audio  feedback. 
Software  and  hardware  Tools.  AudioMath  and 
Theo  &  Seth  were  developed  by  using  Macro¬ 
media  Director  8.5.  In  particular,  AudioMath 
was  developed  with  a  library  of  routines  for  ex¬ 
ternal  joystick  control,  Xtra  RavJoystick.  A  joy¬ 
stick  Side-Winder  was  used  in  conjunction  with 
Force  Feedback,  mouse,  keyboard,  and  Wacon 
tablets. 


Figure  1.  Interfaces  of  AudioMath 
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Interfaces.  The  principal  interfaces  of  Audio- 
Math  for  children  with  residual  vision  are  dis¬ 
played  in  Figure  1 .  For  blind  children  interfaces 
are  only  audio-based.  A  is  the  identification 
screen  with  two  modes:  facilitator  or  student 
(two  buttons).  B  considers  the  level  of  complex¬ 
ity  (list  box),  content  (list  box),  and  input  device 
(buttons).  Content  can  be  filled,  upgraded,  and 
edited  by  using  different  media.  C  is  the  main 
interface  of  AudioMath  and  includes  options 
such  as  the  position  of  the  card  grid,  accumu¬ 
lated  score,  time  elapsed  (through  speech), 
restart,  register,  and  exit  (through  buttons).  D 
includes  a  logging  actual  use  register  (buttons) 
describing  each  game  and  movements. 

Theo  &  Seth  is  game-based  and  includes  inter¬ 
esting  mathematics  learning  activities  with  dif¬ 
ferent  levels  of  complexity.  The  metaphor  used 
resembles  a  grange  with  two  major  virtual  envi¬ 
ronments:  the  kitchen  and  the  henhouse.  Two 
personages  interact  with  the  learner  and  exert 
actions  during  the  game.  The  kitchen  covers 
two  topics:  1.  The  henhouse  is  a  virtual  space 
were  learners  learn  how  to  do  sum  and  rest 
(figure  a),  2.  Introduce  ordinal  numbers  and  to 


Figure  2.  Interfaces  of  Theo  &  Seth 


form  soup  of  numbers  (see  figure  b  and  c),  and 
3.  Cardinality,  including  the  number,  the  posi¬ 
tion  in  the  numerical  straight  line  and  informa¬ 
tion  about  the  antecessor  and  successor  (figure 
d). 

Interaction.  AudioMath  allows  interaction  with 
available  interface  elements  such  as  buttons 
and  text  screens  through  keyboard.  Each  inter¬ 


action  triggers  an  audio  feedback  and  a  high 
visual  contrast  screen  to  be  perceived  by  chil¬ 
dren  with  residual  vision.  The  child  has  to  move 
through  a  grid  and  open  the  corresponding  to¬ 
ken.  Each  cell  has  associated  music  tones  that 
identify  the  position  in  the  grid.  Sound  is  lis¬ 
tened  when  moving  through  cells.  When  open  a 
token  the  associated  element  is  visible  by  trig¬ 
gering  an  audio  feedback.  For  example,  if  the 
image  is  a  car,  a  real  traffic  car  sound  is  trig¬ 
gered.  When  open  a  correct  token  pair  a  feed¬ 
back  is  set.  Finally,  when  all  pairs  are  made  the 
time  used,  the  final  score,  and  feedback  are 
given. 

AudioMath  can  be  interacted  through  keyboard, 
joystick,  and  tablets.  A  few  keystrokes  are  used 
with  the  keyboard.  These  devices  have  been 
used  in  different  applications  developed  by  the 
research  group  after  testing  with  children  with 
visual  disabilities.  The  Microsoft  Sidewinder 
joysticks  in  conjunction  with  Xtra  RavJoystick 
for  Macromedia  Director  allow  grading  the  user 
position  in  the  grid  and  give  direct  feedback 
with  different  forces.  Counter  forces  to  the 
movement  are  generated  per  each  token  posi¬ 
tion  change  as  well  as  vibratory  forces  indicate 
to  be  near  to  the  grid  edge:  up,  down,  left,  and 
right.  Force  Feedback  Joysticks  allow  direct 
interaction  with  diverse  degrees  of  freedom.  A 
plastic  graphic  grid  is  posed  on  the  tablet  defin¬ 
ing  the  position  of  each  token.  A  pen  is  used  to 
point  and  select  interface  elements. 

During  interaction  with  Theo  &  Seth  children 
were  exposed  to  cognitive  tasks.  They  had  to 
solve  diverse  concrete  tasks  that  complement 
and  enrich  the  experience  with  the  virtual  envi¬ 
ronment.  Tasks  involved  exercises  by  using 
simple  concrete  materials  used  in  everyday  life. 
The  way  children  interact  with  the  software  is 
through  keyboard  and  sound.  All  the  interac¬ 
tion  is  thought  in  such  a  way  that  children  can 
navigate  without  assistance.  To  support  this  at 
the  very  beginning  there  is  a  tutorial  to  learn 
how  to  use  the  keyboard  by  orienting  them 
about  the  localization  of  keystrokes  necessary 
to  navigate  the  virtual  world.  Once  embedded  in 
the  environment  children  can  navigate  freely 
and  autonomously  by  interacting  with  different 
interfaces. 

AudioMath  and  Theo  &  Seth  emphasize  learn¬ 
ing  concepts  such  as  establishing  correspon- 
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dence  and  equivalency  relationships,  memory 
development,  and  differentiating  tempo-spatial 
notions.  AudioMath  were  implemented  by  inte¬ 
grating  mathematics  content  based  on  the  cur¬ 
rent  school  curriculum.  We  embedded  the  soft¬ 
ware  with  mathematical  concepts  such  as  posi¬ 
tional  value,  sequences,  additive  decomposi¬ 
tion,  multiplication,  division,  subtraction.  We 
wanted  to  observe  how  audio-based  virtual  en¬ 
vironments  can  foster  the  construction  of 
mathematics  learning  in  the  mind  of  children 
with  visual  disabilities. 

METHODOLOGY 

Participants.  The  study  with  AudioMath  was 
developed  with  ten  children  ages  8  to  15  who 
attend  a  school  for  blind  children  in  Santiago, 
Chile.  The  sample  was  conformed  of  5  girls  and 
5  boys.  Most  of  them  have  also  added  deficits 
such  as  diverse  intellectual  development:  nor¬ 
mal,  slow  normal,  border  line,  below  to  normal, 
and  with  mental  deficit.  Four  special  education 
teachers  also  participated.  All  learners  met  the 
following  prerequisites:  to  be  legally  blind,  to 
know  the  natural  numbers,  to  express  se¬ 
quences  orally,  to  order  numbers,  to  decom¬ 
pose  numbers  through  audio  means,  to  men¬ 
tally  estimate  results  of  additions  and  subtrac¬ 
tions,  to  mentally  determine  products  and  coef¬ 
ficients,  to  mentally  decompose  numbers  in 
additions,  to  manipulate  multiplication  tables 
efficiently,  and  to  have  notions  of  fractions. 

Theo  &  Seth  was  usability  tested  with  nine  7  to 


Figure  3-4.  Children  solving  cognitive 


8  years  old  learners  with  visual  disabilities,  four 
totally  blind  and  five  with  residual  vision.  All  of 
them  are  legally  blind.  Testing  was  applied  in  a 
school  for  blind  children  in  Chile.  We  observed 
learners  responses  to  both  the  use  of  only  au¬ 
dio-based  virtual  environments  and  the  interac¬ 
tion  with  virtual  environments  and  associated 
cognitive  tasks.  Subjects  were  pre  and  post 
tested  on  mathematics  learning. 

Evaluation  Instruments.  Two  measurement 
tests  were  used  to  evaluate  the  impact  of 
AudioMath  on  learning  and  practice  of  mathe¬ 
matical  concepts  such  as  positional  value,  se¬ 
quences,  additive  decomposition,  multiplication, 
and  division:  evaluation  of  mathematics  knowl¬ 
edge  test^^  and  a  usability  evaluation  test  for 
end-users.  The  evaluation  of  mathematics 
knowledge  test  measures:  The  capacity  to  un¬ 
derstand  numbers  (oral  and  written),  skills  nec¬ 
essary  to  make  oral  and  written  calculations, 
skills  to  count  numeric  series  and  graphic  ele¬ 
ments,  and  skills  for  mathematics  reasoning. 

Two  measurement  tests  were  used  to  evaluate 
the  impact  of  Theo  &  Seth:  1.  Precalculus  Test 
(by  Neva  Milicic  and  Sandra  Schmidt)  that 
evaluates  the  development  of  mathematics  rea¬ 
soning  in  students  in  early  primary  school;  2. 
Mathematics  Knowledge  Test  (Benton  and  Lu- 
ria),  that  evaluates  the  mathematics  knowledge 
of  first  grade  students.  The  Mathematics  Knowl¬ 
edge  Test  was  adapted  to  the  specific  needs  of 
learners  with  visual  disability. 


tasks  with  AudioMath  and  Theo  &  Seth 
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Procedure.  Children  were  tested  in  the  school 
from  July  to  November  2003,  twice  a  week,  and 
two  one  hour  sessions  per  week.  They  followed 
the  steps  of  pre-testing  (evaluation  of  mathe¬ 
matics  knowledge  test),  interacting  with  applica¬ 
tions,  solving  cognitive  tasks  (see  Figure  2  and 
3),  and  post-testing  (evaluation  of  mathematics 
knowledge  test).  Interacting  with  AudioMath 
and  solving  cognitive  tasks  were  the  main  steps 
of  the  study.  During  these  steps  children  were 
observed  and  assisted  by  four  special  educa¬ 
tion  teachers  filling  check  lists  and  registering 
observed  behaviors.  They  also  applied  a  usabil¬ 
ity  evaluation  test  for  end-users  developed  by 
the  authors. 

COGNITIVE  IMPACT 

During  the  interactive  sessions  we  realized  that 
mathematical  content  used  was  appropriate  to 
the  educational  level  of  the  sample.  We  ana¬ 
lyzed  the  results  case  by  case  because  the 
sample  was  not  homogeneous  in  key  variables 


Figure  5.  Pretest-Posttest  gains  in  mathematics 
knowiedge 


such  as  cognitive  deficits  and  different  degrees 
of  blindness. 

Children  performed  increasingly  well  in  the 
mathematics  knowledge  test.  An  overall  view  of 
initial  results  shows  pretest  -  posttest  gains  in 
mathematics  knowledge  (see  Figure  5),  thus 
indicating  that  interaction  with  AudioMath  asso¬ 
ciated  with  cognitive  tasks  can  improve  mathe¬ 
matics  learning  in  these  children. 


In  mathematics  learning  the  results  were  prom¬ 
ising.  Most  evaluated  mathematics  content  was 
well  attained  by  learners  with  visual  disabilities. 
The  highest  gains  were  in  oral  calculation 
(75%)  and  countdown  of  numeric  series 
(100%).  We  believe  that  these  results  were  also 
partially  due  to  a  better  attitude  of  the  children 
toward  work  and  mathematics  knowledge  con¬ 
struction  such  as  multiplication  tables  and  use 
of  mathematics  operations.  They  impacted 
positively  on  learners  by  increasing  their  cer¬ 
tainty. 

In  Theo  &  Seth  the  subjects  were  pre  and  post 
tested  on  mathematics  learning.  Preliminary 
results  concerning  the  pre  calculus  test  indi¬ 
cates  that  children  evidence  positive  changes  in 
mathematics  knowledge.  Interesting  results 
were  obtained  when  blind  children  presented 
an  added  deficit  such  as  mental  development. 
In  these  cases  the  learning  of  concepts  such  as 
cardinality  showed  important  pretest-posttest 
gains. 

Our  results  indicate  that  learners  with  more 
mathematics  learning  gains  were  those  that 
participated  by  solving  associated  cognitive 
tasks.  Thus  we  can  conclude  that  the  audio- 
based  virtual  environment  by  itself  does  not 
have  an  isolated  impact  on  mathematics  learn¬ 
ing.  Concrete  cognitive  learning  tasks  are  pow¬ 
erful  supports  to  the  interaction  with  the  audio 
interfaces.  These  children  have  not  formal  ab¬ 
stract  thinking  that  is  somehow  embedded  in 
the  software.  Then  concrete  tasks  are  neces¬ 
sary  to  accompany  interaction  experiences  with 
audio-based  interactive  environments. 

At  the  beginning  most  children  did  not  even 
know  the  space  distribution  of  keystrokes  in  a 
keyboard.  At  the  end  of  the  experience  they 
used  software  commands  efficiently.  A  tutorial 
included  may  have  helped  them  in  this  task. 

Colors  used  in  the  graphical  interface  such  as 
yellow  and  blue  produced  a  good  contrast  for 
children  with  residual  vision.  However  the  varia¬ 
tions  types  of  the  same  color  used  did  not  allow 
differentiating  one  element  to  the  other  and  thus 
children  with  low  vision  got  confused. 

Audio  stimuli  such  as  sound  of  bird’s  whispers, 
personages,  tools  and  devices  of  the  kitchen 
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Figure  6-7  Pretest-Posttest  gains  in  mathematics  knowledge 


motivated  and  help  them  to  interact  with  the  soft¬ 
ware  more  actively.  We  have  also  found  that  to 
attain  meaningful  learning  sound-based  environ¬ 
ments  should  be  accompanied  with  cognitive 
tasks  involving  written  number  recognition  and 
reproduction. 

Finally,  we  can  conclude  that  thanks  to  the  in¬ 
teraction  with  AudioMath  and  Theo  &  Seth,  and 
the  associated  cognitive  tasks  learners  with 
visual  abilities  developed  mathematics  skills. 
This  is  a  major  result  in  our  research  because 
we  are  initially  observing  that  audio-based  vir¬ 
tual  environments  can  foster  the  construction  of 
mathematics  learning  in  the  mind  of  children 
with  visual  disabilities. 

DISCUSSION 

We  have  introduced  AudioMath  and  Theo  & 
Seth,  virtual  environments  designed  to  enhance 
memory  and  mathematics  skills  in  children  with 
visual  disabilities  through  audio.  The  software 
was  made  by  and  for  children  with  visual  dis¬ 
abilities.  They  participated  actively  in  the  devel¬ 
opment  of  the  software.  We  have  also  designed 
interfaces  for  both  blind  and  children  with  resid¬ 
ual  vision.  A  usability  study  was  implemented 
with  end-users,  facilitators,  observers,  and  ex¬ 
perts. 

AudioMath  and  Theo  &  Seth  were  highly  ac¬ 
cepted  by  end-users.  They  liked,  enjoyed,  and 
were  motivated  when  interacted  with  the  soft¬ 
ware.  The  flexibility  of  this  application  is  also  a 
plus.  Teachers,  children,  and  parents  can  in¬ 
clude  new  objects  and  sounds  to  adapt  them  to 
their  needs.  Thus,  children  with  visual  disabili¬ 


ties  can  choose  sounds  to  be  interacted  with 
and  embed  them  into  AudioMath  and  Theo  & 
Seth.  Content  can  be  changed  and  updated 
easily.  Both  virtual  environments  can  be  used 
to  support  learning  primary  school  mathemat¬ 
ics. 

The  use  of  concrete  materials  was  also  a  plus 
in  this  study.  The  children’s  understanding  was 
easier  when  they  first  interacted  with  concrete 
materials  and  then  with  AudioMath  and  Theo  & 
Seth.  Parallel  interaction  with  both  concrete 
material  and  AudioMath/Theo  &  Seth  was  also 
an  advantage.  Once  they  developed  their  own 
mental  model  of  the  software  the  interaction 
was  enriched. 

Force  Feedback  Joysticks  introduced  a  new 
scenario  in  virtual  environment  for  blind  chil¬ 
dren.  They  can  provide  information  and  tactile 
sensations  through  force  feedback.  This  can 
help  to  decrease  audio  stimuli  and  relief  possi¬ 
ble  acoustic  pollution.  Joysticks  are  devices 
with  a  high  potential  of  use  due  to  the  availabil¬ 
ity  of  many  buttons. 

Our  model  fits  well  the  learning  of  primary 
school  mathematics  concepts  such  as  posi¬ 
tional  value,  sequences,  additive  decomposi¬ 
tion,  multiplication,  and  division.  Children  per¬ 
formed  increasingly  well  in  the  mathematics 
knowledge  test.  Oral  calculation  and  countdown 
numeric  series  were  highly  achieved  as  well  as 
numeric  and  associative  memory.  Concrete 
cognitive  tasks  were  crucial  in  this  achieve¬ 
ment.  We  firmly  conclude  that  interaction  with 
AudioMath  and  Theo  &  Seth  associated  with 
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cognitive  tasks  can  help  to  improve  mathematic 
learning  in  these  children. 

More  qualitative  data  are  being  analyzed.  Most 
of  them  are  case  study  because  each  child  with 
visual  disabilities  is  a  whole  case  that  deserves 
a  deep  analysis  to  construct  meaning  about  the 
role  that  can  play  audio-based  devices  in  learn¬ 
ing  general  and  specific  domain  skills. 

Finally,  we  are  convinced  that  further  research 
studies  we  are  implementing  right  now  concern¬ 
ing  mathematic  learning  will  reaffirm  our  hy¬ 
pothesis  that  audio-based  virtual  environment 
can  foster  the  construction  of  mathematics 
learning  in  the  mind  of  children  with  visual  dis¬ 
abilities. 
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Abstract:  In  vivo  exposure  is  a  well-tested  and  effective  treatment  for  anxious  and  phobic  disorders. 
However,  it  is  not  viable  to  expose  patients  with  War  PTSD  to  real  war  environments.  As  virtual  real¬ 
ity  exposure  (VRE)  is  currently  the  most  approximate  representation  of  a  real  war  situation,  we  hy¬ 
pothesized  that  a  treatments  of  War  PTSD  that  included  VRE  along  with  Drug  Treatment  (DT)  would 
be  better  than  DT  alone.  The  clinical  population  of  this  ongoing  research  consists  of  male  subjects 
with  the  diagnostic  of  War  PTSD  according  to  DSM-IV-TR  (APA,  2001)  who  looked  for  treatment  at 
Hospital  Julio  de  Matos  in  Lisbon,  Portugal.  The  patients  were  distributed  into  3  treatments  groups 
(VRE;  Drug  treatment;  VRE+Drug  Treatment).  The  adequate  therapeutic  dosage  of  Sertraline 
(Zoloft,  Pfeizer)  will  be  administrated  during  16  weeks  to  the  Drug  Treatment  groups.  VRE  groups 
will  use  a  Head  Mounted  Device  that  enables  fully  immersive  experience  in  the  following  war  virtual 
scenarios:  mine  deflagration,  mine  deflagration  +  ambush,  ambush  and  assisting  casualties  and 
waiting  for  a  rescue  helicopter.  These  scenarios  were  developed  using  the  most  up-to-date  video 
game  graphics  engine.  It  is  also  a  goal  of  this  study  to  compare  different  measurement  procedures. 
In  this  way,  CAPS,  BDI,  STAI,  SCL-90,  MCM-II  for  psychometric  measures  and  TAS,  DES,  PQ, 
SUDS,  heart  rate  and  blood  pressure,  ECG,  EEG  and  ACTH  for  physiological  measures  are  the 
evaluation  procedures  selected  for  assessing  the  results. 


INTRODUCTION 

Studies  published  over  the  past  thirty  years 
show  that  exposure  to  phobic  stimulus  is  a  fun¬ 
damental  technique  in  helping  patients  get  be¬ 
yond  difficulties  and  minimizing  the  personal, 
professional  and  social  impact  of  many  anxiety 
disorders.  Virtual  reality  exposure  (VRE)  hs 
been  used  since  1 992  to  treat  these  disorders  - 
especially  acrophobia,  fear  of  flying,  fear  of 
speaking  in  public,  fear  of  driving,  etc. 

Virtual  reality  exposure  (VRE)  is  more  efficient 
than  conventional  imagination  exposure  (IE). 
For  one,  subjects  can  have  difficulties  in  creat¬ 
ing  or  visualizing  anxiety-invoking  mental  im¬ 
ages. Further,  VRE  allows  for  a  more  real 
reproduction^ and  is  more  immersive  than 
imagination  exposure.'*  ®'®  VRE  induces  concen¬ 
tration/attention  augmentation  to  the  phobic 
situation  and,  during  phobic  stimulus  exposure, 
diminishes  the  probability  of  distraction  or  cog¬ 
nitive  avoidance  to  the  feared  situation.  In  this 
context,  Vincelli®  states  that  virtual  reality  tech¬ 
nology  produces  an  experience  capable  of  di¬ 


minishing  the  gap  between  reality  and  imagina¬ 
tion. 

Despite  having  established  that  VRE  may 
gather  some  advantages  over  traditional  psy¬ 
chotherapeutic  measures  (namely,  imagination 
exposure  and  in  vivo  exposure),  studies  about 
the  application  of  VRE  in  anxiety  disorders  that 
generally  require  the  administration  of  psy- 
chopharmacological  procedures  are  lacking.® 
Such  is  the  case  with  war  PTSD.  As  a  disorder 
with  particularly  profound  effects  on  patient,  his/ 
her  family  structure  and  on  society  as  a  whole, 
psychotherapeutic  measures  are  usually  not 
sufficient  to  ease  the  distress  caused  by  the 
memory  of  war  episodes.  More  often  than  not, 
such  patients  follow  a  therapy  that  includes 
both  psychotherapeutic  procedures  and  the 
administration  of  psychopharmacological  drugs 
such  as  sertraline.  This  is  the  case  of  thou¬ 
sands  of  Portuguese  males.  Between  1961  and 
1974,  the  Portuguese  fought  a  war  on  several 
fronts  (Mozambique,  Angola  and  Guine  Bis- 
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sau),  the  majority  of  them  spending  four  years 
on  tour  in  the  jungle.  Without  proper  epidemiol¬ 
ogical  studies,  authors  estimate  that  25.000  of 
them  still  suffer  from  war  PTSD.  As  in  many 
cases  a  composed  therapy  of  drugs  and  imagi¬ 
nation  exposure  doesn’t  show  significant  re¬ 
sults.  We  therefore  hypothesized  that  the  expo¬ 
sure  to  virtual  war  scenarios  in  addition  to  drug 
treatment  would  provide  a  higher  reduction  on 
PTSD  symptomatology.  Rothbaum  et 
studied  VRE  as  an  alternative  treatment  to 
imagination  exposure  (IE)  in  Vietnam  war  veter¬ 
ans  with  PTSD,  though  no  matching  of  VRE 
with  drug  treatment  was  made.  The  patients 
were  exposed  to  two  virtual  scenarios:  a)  a 


Huey  helicopter  flying  over  a  virtual  Vietnam 
field  and  b)  a  clearing  surrounded  by  jungle. 
The  authors  of  said  study  write  that  therapists 
found  a  reduction  of  PTSD  symptoms  in  34%  of 
the  subjects  and  a  diminishing  of  PTSD  symp¬ 
tomatology  in  45%  of  them.  These  gains  were 
maintained  throughout  6  months  of  follow-up. 
However,  results  of  existent  studies  show  that 
more  research  is  still  needed. 

Another  goal  of  the  present  ongoing  research  is 
to  define  which  assessment  methodology 
should  be  used  for  monitoring  treatment  effects. 
In  most  of  the  studies  only  psycometric  meas¬ 
ures  were  used.  In  fact,  Wlhelm  &  Roth^^ 


Picture  2a)  Mine  deflagration  +  ambush  Picture  2b)  Facing  fellow  soldiers  death/injury 

situations  and  waiting  for  causalities  evacuation 
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stated  that  psycophysiological  methods  are  not 
usual  to  evaluate  the  responses  to  a  certain 
aversive  stimuli.  Nevertheless,  it  is  accepted 
that  self  reports  are  not  able  to  give  the  com¬ 
plete  picture  of  the  reactions  and  modifications 
endured  by  the  subject  when  exposed  to  the 
stimuli.  We  therefore  propose  that  besides  psy¬ 
chometric  measures  like  CAPS,  BDI,  STAI, 
SCL-90,  MCM-II,  physiological  procedures  such 
as,  heart  rate  and  blood  pressure,  ECG,  EEG 
and  ACTH,  should  be  used  to  assess  the  expo¬ 
sure  results. 

METHODOLOGY 

VR  Model 

For  the  virtualization  of  traumatic  scenes  from 
the  war  of  Portuguese  ex-colonies  in  Africa 
(Guine  Bissau,  Angola,  Mozambique),  the 
following  scenarios  were  chosen  (see  picture 
1  a),  1  b),  2  a)  and  2  b)): 

a)  Mine  deflagration; 

b)  Ambush; 

c)  Mine  deflagration  +  Ambush; 

d)  Facing  fellow  soldiers  death/injury 
situations  and  waiting  for  causalities 
evacuation. 

The  scenarios  were  developed  at  the  Psychol¬ 
ogy  Computing  Laboratory  of  University  Luso- 
fona  (Lisbon,  Portugal)  using  a  Half  Life  2 
graphic  engine.  For  the  immersive  exposure  a 
last  generation  fully  immersive  Head  Mounted 
Display  HiRes  800  PC  3D  with  an  Intertrax  Iner¬ 
tia  Cubes  (3DOF)  will  be  used. 

Clinical  Protocol 

Procedures 

90  selected  subjects  will  be  included,  in  se¬ 
quential  order,  in  three  groups  (30  subjects 
each); 

•  Virtual  Reality  Exposure  (VRE)  Treatment 
consists  in  30  minutes  sessions  for  14 
weeks.  Our  option  was  presented  by 
North. ^  The  VRE  session  will  be  interrupted 
if  a  patient  feels  sick.  During  the  session 
the  patient  will  be  sitting  (per  suggestion  by 
North  et  a/,^).  The  Patient  will  be  instructed 
that  he  may  not  speak  about  war  experi¬ 
ences  before  beginning  the  session. 


•  Drug  Treatment  (DT) 

•  Prescription  will  be  executed  by  a  psychia¬ 
trist  with  clinical  experience  who  will  indi¬ 
cate  the  adequate  therapeutic  dose 
(initiating  with  50  mg/day  and  after  fixating 
the  doses  in  100  or  200  mg/day,  as  re¬ 
quired).  Sertralina  (Zoloft,  Pfizer)  will  be  the 
drug  treatment  to  be  administered  during 
16  weeks. 

•  Drug  Treatment  +  Virtual  Reality  Exposure 
(DT+VRE) 

•  Same  clinical  intervention  concerning  thera¬ 
peutic  relationship,  techniques,  frequency 
and  duration  of  therapeutic  sessions  of 
VRE  and  DT  will  be  upheld.  The  difference 
consists  only  on  the  exposure  technique 
administered. 

VRE  groups  will  receive  psychoeducational  in¬ 
formation  about  anxiety  physiology  and  cogni¬ 
tive-behavioral  models  of  PTSD  and  anxiety 
management  techniques  (diaphragmatic  respi¬ 
ration  and  relaxation)  during  the  two  first  ses¬ 
sions.  Therapist  will  help  the  patient  modify  his 
dysfunctional  thoughts  and  beliefs  through  cog¬ 
nitive  restructuring. 

To  be  included  in  DT  and  DT+VRE  groups,  all 
subjects  that  are  taking  medication  will  have  to 
pass  a  15  day  period  of  wash-out. 

Subjects 

Inclusion  criteria  are  male  subjects  with  the  di¬ 
agnostic  of  War  PTSD  according  to  DSM-IV-TR 
(APA,  2001)  who  attend  a  Lisbon  central  psy¬ 
chiatric  hospital  (Hospital  Julio  de  Matos).  An 
independent  technician  will  evaluate  subjects 
through  a  semi-structured  clinical  interview. 
Exclusion  criteria  are  cardiovascular  disease, 
epilepsy,  substance  dependence  (in  the  last 
year). 

After  complete  description  of  the  study  to  the 
subjects,  written  informed  consent  will  be  ob¬ 
tained  in  accordance  with  Helsinki  Declaration 
of  1975,  as  revised  in  1983.  Patients  that  ac¬ 
cept  to  participate  in  the  study  will  be  sequen¬ 
tially  distributed  accordingly  experimental 
groups  up  to  30  persons  for  group. 
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Assessment 

There  will  be  a  pre-treatment  evaluation  period 

and  an  evaluation  at  1,  2,  4, 12  e  24  months. 

Pre  -  2  -  4  -  6  -12  -  24  months 
Evaluating  measures  are: 

PTSD 

•  CAPS  -  Clinician-Administered  PTSD 
Scale^^ 

A  structured  clinical  interview  that  allows 
for  the  evaluation  17  symptoms  (frequency 
and  intensity)  and  8  factors  associated  with 
social  and  professional  functioning  as  well 
as  clinical  progress  during  treatment. 

•  lES  -  Impact  of  Event  Scale^"* 

A  15  items  self-report  questionnaire  evalu¬ 
ating  experiences  of  avoidance  and  intru¬ 
sion  giving  current  subjective  distress  re¬ 
lated  to  a  specific  life  event. 

Psvchopatholoqic  Comorbilitv 

•  BDI  -  Beck  Depression  Inventory^® 

A  21  questions  survey  presented  in  multiple 
choice  format  which  purports  to  measure 
presence  and  degree  of  depression  by  as¬ 
sessing  a  specific  symptom  or  attitude 
"which  appear(s)  to  be  specific  to  de¬ 
pressed  patients,  and  which  are  consistent 
with  descriptions  of  the  depression  con¬ 
tained  in  the  psychiatric  literature."^^ 

•  STAI  -  State-Trait  Anxiety  Inventory^® 

A  self-report  inventory  compose  by  two 
subscales  developed  to  measure  state  and 
trait  anxiety  with  20  items  each. 

•  SCL-90  -  Symptom-Check-List-90- 
Revised^^ 

A  brief,  multidimensional  self-report  inven¬ 
tory  designed  to  screen  for  a  broad  range  of 
psychological  problems  and  symptoms  of 
psychopathology.  It  is  composed  by  9  symp¬ 
tom  scales  -  Somatization,  Obsessive- 
Compulsive,  Interpersonal  Sensitivity,  De¬ 
pression,  Anxiety,  Hostility,  Phobic  Anxiety, 
Paranoid  Ideation  and  Psychoticism  -  and  3 
global  indices  -  Global  Severity  Index,  Posi¬ 
tive  Symptom  Distress  Index  and  Positive 
Symptom  Total.  The  SCL-90-R  instrument  is 


also  useful  as  a  progress  or  outcomes 
measurement  instrument. 

Personality 

•  MCMI-II  -  Millon  Clinical  Multiaxial  Inven- 
tory-lT® 

175  true/false  items  grouped  in  10  clinical 
personality  pattern  scales,  3  severe  person¬ 
ality  pathology  scales,  6  clinical  syndrome 
scales,  3  modifier  indices  and  1  validity  in¬ 
dex.  The  MCMI-II  instrument  has  been  re¬ 
vised  to  correspond  with  DSM-IV  criteria. 

VR  reactions 

•TAS  -  Tellegen  Absorption  Scale^® 

The  most  commonly  used  measure  of  ab¬ 
sorption  is  the  Tellegen  Absorption  Scale 
(TAS),  a  34  true/false  items  that  evaluate 
subject  involvement  capacity  on  the  task  or 
environment.  The  TAS  represents  one  of 
11  primary  personality  dimensions  meas¬ 
ured  by  the  Multidimensional  Personality 
Questionnaire  (MPQ^®). 

•DES  -  Dissociative  Experiences  Scale^° 

It’s  a  28  item  scale  of  self-report  evaluating 
dissociative  experiences.  It  is  the  only  dis¬ 
sociative  instrument  that  has  been  subject 
to  a  number  of  replication  studied  by  inde¬ 
pendent  investigators  (Ross,  Norton  & 
Anderson,  1988). 

•PQ  -  Presence  Questionnaires^^ 

A  subjective  questionnaire  in  which  subjects 
classify  their  experience  to  VR  concerning 
control,  body  sensations,  distraction  and 
reality  factors  in  a  7  points  Likert  scale. 

•SUDS  -  Subjective  Units  of  Discomfort  (0-100 

-LB-10mn-20  mn) 

This  scale  measure  the  degree  in  which  the 
subject  is  affected  by  VR  by  reporting  the 
anxiety  level  experienced  during  the  expo¬ 
sition  session  in  regular  intervals.  Scores 
range  from  0  to  100  (0  -  no  anxiety;  100  - 
maximum  anxiety/panic  attack). 

•Verbal  and  motor  behavior  records,  during  the 

session  of  VRE^ 

•Motivation  and  control  sensation  at  the  end 

of  the  VRE  session^ 

•Physiological  measures 

Patients  will  be  evaluated  according  to  their 
medical  history  and  to  their  records  of  heart 
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rate  and  blood  pressure,  EGG  e  EEG  and 
ACTH,  Cortisol  and  Prolactine  analysis. 

3.  FINAL  CONSIDERATIONS 

The  present  paper  continues  the  methodologi¬ 
cal  thread  of  the  on-going  research  on  the  use 
of  virtual  reality  exposure  (VRE)  as  a  therapeu¬ 
tic  procedure  on  a  clinical  population  of  war 
PTSD  veterans.  The  study  population  is  repre¬ 
sented  by  90  subjects  enrolled  in  the  PTSD 
department  of  the  Hospital  Julio  de  Matos.  It  is 
expected  that  VRE  when  associated  with  a  psy- 
chopharmacological  drug  will  reduce  PTSD 
symptomatology.  Moreover,  it  is  also  expected 
that  the  application  of  physiological  measures 
will  strengthen  the  assessment  of  VRE  results. 
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Clinical  Observations 


A  Progress  Report  of  Long-Term  Robot  Assisted  Activity 
at  a  Health  Service  Facility  for  the  Aged 
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Abstract;  We  have  proposed  Robot-Assisted  Therapy  and  Activity  since  1996,  and  have  been  devel¬ 
oping  mental  commit  robots  that  provide  psychological,  physiological,  and  social  effects  to  human 
beings  through  physical  interaction^^^K  The  appearances  of  these  robots  look  like  real  animals  such 
as  cat  and  seal.  The  seal  robot,  Paro,  was  developed  especially  for  therapy.  We  have  applied  seal 
robots  to  therapy  of  children  at  pediatric  hospitals''^  and  to  assisting  activity  of  elderly  people  at  a  day 
service  center°^''K  Recently,  several  research  groups  have  tried  robot  assisted  therapy  and  activity. 
Dautenhahn  has  used  mobile  robots  and  robotic  dolls  for  therapy  of  autistic  children  Besides,  ro¬ 
bot-assisted  activity  that  uses  commercialized  animal  type  robots  (such  as  AIBO"\  NeCoRo,  etc.) 
has  been  tried'^^''''.  For  example,  Yokoyama  used  AIBO  in  a  pediatrics  ward,  and  observed  the  inter¬ 
action  between  children  and  pointed  out  that  the  initial  stimulus  received  from  AIBO  was  strong. 
However,  the  long  term  stability  was  quite  weak,  compared  with  living  animals'^K  In  this  presentation, 
we  wiil  explain  the  results  of  the  robot-assisted  activity  for  elderly  people  at  a  health  service  facility 
for  the  aged  for  more  than  one  year. 

Method:  In  order  to  investigate  the  effects  of  seal  robots  to  the  elderly  people,  we  evaluated  moods 
of  elderly  people  by  face  scales' that  express  person’s  moods  by  Illustration  of  person’s  faces, 
questionnaires  of  Geriatric  Depression  Scales  (GDS)'^K  Seal  robots  were  provided  into  the  health 
service  facility  on  two  days  per  a  week  from  Aug.  2003. 

Results:  The  results  efface  scale  and  GDS  showed  that  feelings  and  depression  of  elderly  people 
were  improved  by  interaction  with  the  seal  robots,  Paro.  Regarding  a  case  study,  Hanako 
(pseudonym),  aged  89,  was  sociable  and  comparatively  independent.  On  the  first  day  of  the  interac¬ 
tion  with  Paro,  she  looked  a  little  nervous  of  the  experiment.  However,  she  soon  came  to  like  Paro. 
She  treated  Paro  like  her  child  or  grandchild.  Her  face  scale  scores  after  interaction  were  always 
lower  than  before  interaction  after  the  first  day.  Unfortunately,  she  was  hospitalized  during  Dec.  10  to 
26,  2003.  When  she  met  Paro  for  the  first  time  after  leaving  hospital,  she  said  to  Paro  "I  was  lonely, 
Paro.  I  wanted  to  see  you  again. "  Her  GDS  score  then  improved.  To  the  present,  she  has  continued 
to  Join  the  activity  and  willingly  Interacted  with  Paro.  Caregivers  commented  that  interaction  with  Paro 
made  the  people  laugh  and  become  more  active.  For  example,  their  facial  expression  changed,  sof¬ 
tened,  and  brightened.  In  addition,  Paro  encouraged  the  people  to  communicate,  both  with  each 
other  and  caregivers,  by  becoming  their  common  topic  of  conversation.  Thus,  the  general  atmos¬ 
phere  became  brighter. 

Conclusions:  We  have  used  seal  robots,  Paro  in  RAA  for  elderly  people  at  a  health  service  facility  for 
the  aged  since  August  2003.  The  results  showed  that  interaction  with  Paro  improved  their  moods  and 
depression,  and  then  the  effects  showed  up  through  more  than  one  year.  Consequently,  the  seal  ro¬ 
bots,  Paro  were  effective  for  therapy  at  health  service  facilities. 
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BACKGROUND 

Interaction  with  animals  has  long  been  known  to 
be  emotionally  beneficial  to  people.  The  effects 
of  animals  on  humans  have  been  applied  to 
medical  treatment.  Especially  in  the  United 
States,  animal-assisted  therapy  and  activities 
(AAT&AAA)  are  becoming  widely  used  in  hospi¬ 
tals  and  nursing  homes.^^AAT  has  clear  goals 
set  out  in  therapy  programs  designed  by  doc¬ 
tors,  nurses  or  social  workers,  in  cooperation 
with  volunteers.  In  contrast,  AAA  refers  to  pa¬ 
tients  interacting  with  animals  without  particular 
therapeutic  goals,  and  depends  on  volunteers. 
AAT  and  AAA  are  expected  to  have  3  effects: 

•  Psychological  effect  (e.g.,  relaxation, 
motivation) 

•  Physiological  effect  (e.g.,  improvement  of 
vital  signs) 

•  Social  effect  (e.g.,  stimulation  of  communi¬ 
cation  among  inpatients  and  caregivers) 

However,  most  hospitals  and  nursing  homes, 
especially  in  Japan,  do  not  accept  animals, 
even  though  they  admit  the  positive  effects  of 
AAT  and  AAA.  They  are  afraid  of  negative  ef¬ 
fects  of  animals  on  human  beings,  such  as  al¬ 
lergy,  infection,  bites,  and  scratches. 

We  have  proposed  Robot-Assisted  Therapy  and 
Activity  (RAT  and  RAA)  since  1996,  and  have 
been  developing  mental  commit  robots  that  pro¬ 
vide  psychological,  physiological,  and  social 
effects  to  human  beings  through  physical  inter¬ 
action.^'”  The  appearances  of  these  robots  look 
like  real  animals  such  as  cat  and  seal.  The  seal 
robot,  Paro,  was  developed  especially  for  ther¬ 
apy.  We  have  applied  seal  robots  to  therapy  of 
children  at  pediatric  hospitals^  and  to  assisting 
activity  of  elderly  people  at  a  day  service  cen¬ 
ter.^'®  Recently,  several  research  groups  have 
tried  robot  assisted  therapy  and  activity.  Dau- 
tenhahn  has  used  mobile  robots  and  robotic 
dolls  for  therapy  of  autistic  children. Besides, 
robot-assisted  activity  that  uses  commercialized 
animal  type  robots  (such  as  AIBO,^^  NeCoRo, 
etc.)  has  been  tried. For  example,  Yoko- 
yama  used  AIBO  in  a  pediatrics  ward,  and  ob¬ 
served  the  interaction  between  children  and 
pointed  out  that  the  initial  stimulus  received 
from  AIBO  was  strong.  However,  the  long  term 
stability  was  quite  weak,  compared  with  living 
animals.” 


In  this  paper,  we  discuss  the  application  of  the 
seal  robots  to  assist  elderly  people  at  a  health 
service  facility  for  the  aged,  observing  their  psy¬ 
chological  and  social  effects  for  more  than  one 
year.  Chapter  II  describes  a  seal  robot  that  was 
used  for  robot-assisted  activity  (RAA),  and  ways 
of  experiments.  Chapter  III  explains  the  effects 
of  RAA  on  elderly  people.  Finally,  chapter  IV 
offers  conclusions. 

METHOD 

The  robot  and  its  major  functions  are  shown  in 
Fig.1.  Its  appearance  was  designed  using  a 
baby  harp  seal  as  a  model,  and  its  surface  was 
covered  with  pure  white  fur.  A  newly-developed 
plane  tactile  sensor”  was  inserted  between  the 
hard  inner  skeleton  and  the  fur  to  create  a  soft, 
natural  feel  and  to  permit  the  measurement  of 
human  contact  with  the  robot.  The  robot  is 
equipped  with  the  four  primary  senses:  sight 
(light  sensor),  audition  (determination  of  sound 
source  direction  and  speech  recognition),  bal¬ 
ance  and  the  above-stated  tactile  sense.  Its 
moving  parts  are  as  follows:  vertical  and  hori¬ 
zontal  neck  movements,  front  and  rear  paddle 
movements  and  independent  movement  of 
each  eyelid,  which  is  important  for  creating  fa¬ 
cial  expressions.  The  robot  operates  by  using 
the  3  elements  of  its  internal  states,  sensory 
information  from  its  sensors  and  its  own  diurnal 
rhythm  (morning,  daytime,  and  night)  to  carry 
out  various  activities  during  its  interaction  with 
people. 

INSTRUCTIONS:  The  faces  above  range  from 
very  happy  at  the  left  to  very  sad  at  the  right. 


Figure  1.  Seal  Robot  “Paro” 
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Figure  2.  Face  Scale 


Check  the  face  which  best  shows  the  way  you 
have  felt  inside  now. 

In  order  to  investigate  the  effects  of  seal  robots 
to  the  elderly  people,  we  evaluated  moods  of 
elderly  people  by  face  scales^^  that  express  per¬ 
son’s  moods  by  illustration  of  person’s  faces, 
questionnaires  of  Geriatric  Depression  Scales 
(GDS).^®  The  original  Face  Scale  contains  20 
drawings  of  a  single  face,  arranged  in  serial  or¬ 
der  by  rows,  with  each  face  depicting  a  slightly 
different  mood  state.  However,  sometimes  sub¬ 
jects  are  confused  by  original  face  scale  be¬ 
cause  it  has  too  many  similar  images.  Then,  we 
simplified  it  by  using  seven  images  #1,  4,  7,  10, 
13, 16,  and  19  in  original  ones  (Fig.2). 

Seal  robots  were  provided  into  the  health  ser¬ 
vice  facility  on  two  days  per  a  week  from  Aug. 
2003. 


Figure  3.  An  Elderly  Person  Kissing  the  Seal 
Robot 


RESULTS 

The  elderly  interacted  with  Paro  willingly  from 
the  first  day,  speaking  to  it,  stroking  and  hug¬ 
ging  it.  Sometimes,  they  kissed  it  with  smile 
(Fig. 3).  Paro  became  common  topics  among 
the  elderly  people  and  caregivers  (Fig.4).  They 
talked  about  its  appearance,  kinds  of  animals, 
moods,  and  so  on.  For  example,  “its  eyes  so 
big,”  “it  looks  sleepy,”  etc.  The  elderly  people 
came  to  love  Paro  very  much  and  gave  them 
new  name  “Maru”  and  “Maro,”  respectively.  Af¬ 
ter  3  months  of  the  introduction,  we  added  one 
more  Paro  to  the  facility  because  many  other 
elderly  joined  in  the  activity  voluntarily.  The  new 
Paro  was  given  new  name  “Hana-chan”  by  the 
elderly,  soon.  Moreover,  Paro  have  been  ac¬ 
cepted  by  caregivers  widely.  They  made  a 
home  of  Paro  in  the  facility. 

We  obtained  the  face  scale  data  from  8  subjects 
(Fig.5).  The  average  scores  before  interaction 
varied  from  3.3  to  2.0  through  5  months.  How¬ 
ever,  scores  after  interaction  were  almost  al¬ 
ways  lower  than  those  before  interaction  in 


Figure  4.  Interaction  among  Elderly  People,  a 
Caregiver  and  a  Seal  Robot 
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each  week  (except  Nov.  29).  Especially,  a  sta¬ 
tistically  significant  difference*  was  shown  in 
Nov.  15  (Wilcoxon’s  sign  rank  sum  test:  p*  < 

0.05).  6 


Regarding  a  case  study,  Hanako  (pseudonym), 
aged  89,  was  sociable  and  comparatively  inde¬ 
pendent.  On  the  first  day  of  the  interaction  with 
Paro,  she  looked  a  little  nervous  of  the  experi¬ 
ment.  However,  she  soon  came  to  like  Paro. 
She  treated  Paro  like  her  child  or  grandchild. 
Her  face  scale  scores  after  interaction  were  al¬ 
ways  lower  than  before  interaction  after  the  first 
day  (Fig. 6).  Unfortunately,  she  was  hospitalized 
during  Dec.  10  to  26,  2003.  When  she  met  Paro 
for  the  first  time  after  leaving  hospital,  she  said 
to  Paro  "I  was  lonely,  Paro.  I  wanted  to  see  you 
again."  Her  GDS  score  then  improved  (Fig. 7). 
To  the  present,  she  has  continued  to  join  the 
activity  and  willingly  interacted  with  Paro.  Care¬ 
givers  commented  that  interaction  with  Paro 
made  the  people  laugh  and  become  more  ac¬ 
tive.  For  example,  their  facial  expression 
changed,  softened,  and  brightened.  In  addition, 
Paro  encouraged  the  people  to  communicate, 
both  with  each  other  and  caregivers,  by  becom¬ 
ing  their  common  topic  of  conversation.  Thus, 
the  general  atmosphere  became  brighter. 


Figure  6.  Change  of  Face  Scale  Scores  of  a 
Subject  for  16  months 
(Score:  1=best  mood,  7=worst  mood) 


Before 

iilcrsciioii 

•*- After 
iileracfBn 


Figure  5.  Change  of  Average  Face  Scale 
Scores  of  8  Elderly  People  for  5  Months 
(Score:  1=best  mood,  7=worst  mood) 


Figure  7.  Change  of  GDS  Scores  of  a  Subject 
for  16  months 

(Score:  healthy  condition  <=  5  <  probable  de¬ 
pression) 
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CONCLUSIONS 

We  have  used  seal  robots,  Paro  in  RAA  for  eld¬ 
erly  people  at  a  health  service  facility  for  the 
aged  since  August  2003.  The  results  showed 
that  interaction  with  Paro  improved  their  moods 
and  depression,  and  then  the  effects  showed  up 
through  more  than  one  year.  Consequently,  the 
seal  robots,  Paro  were  effective  for  therapy  at 
health  service  facilities. 
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Abstract;  Background/problem;  The  efficacy  of  VR  exposure  for  the  treatment  of  fear  of  flying  was 
confirmed  in  more  than  8  controlled  clinical  trials.  Important  goals  of  future  studies  are  the  evalua¬ 
tion  of  underlying  treatment  processes  and  the  identification  of  crucial  treatment  components,  both 
in  order  to  optimize  the  treatment.  The  present  study  was  designed  to  examine  the  effect  of  motion 
simulation  on  subjective  and  physiological  fear  reactions  as  part  of  a  VR  exposure  that  also  included 
visual  and  acoustic  stimuli.  Our  hypotheses  were  that  motion  simulation  results  in  stronger  initial 
subjective  and  physiological  fear  reactions,  and  as  a  consequence  is  associated  with  stronger  ha- 
bituations  of  fear  responses  within  VR  flights  and  between  VR  flights.  Furthermore,  we  assumed 
that  motion  simulation  enhances  treatment  efficacy  as  measured  in  the  fear  of  flying  scale  (FFS). 

Methods  and  tools:  Twenty  fife  participants  with  flight  phobia  received  a  virtual  reality  exposure 
treatment  including  written  information  about  fear  of  flying  and  how  to  cope  with  it  (information  book¬ 
let),  one  hour  of  cognitive  therapy,  and  VR  exposure  (four  flights).  However,  VR  exposure  included 
for  twelve  participants  motion  simulation  while  thirteen  participants  received  VR  exposure  without 
motion  simulation. 

Virtual  flights  consisted  out  of  a  different  flying  phases  (start,  flying,  turbulences,  landing)  and  were 
simulated  using  a  head  mounted  display  (HMD),  a  head  tracking  device  and  a  motion  platform  to 
simulate  accelerations  and  provide  proprioceptive  stimuli. 

Results:  Overall,  subjective  fear  ratings  as  well  as  skin  conductance  responses  confirmed  substan¬ 
tial  fear  of  both  groups  during  VR  exposure.  However,  these  responses  were  substantially  stronger 
and  habituated  slower  in  the  VR  motion  group  compared  to  the  VR  no-motion  group.  Nevertheless 
and  in  contrast  to  network  theories  -  which  suggest  that  stronger  activation  of  fear  networks  should 
result  in  an  enhanced  treatment  outcome  -  we  found  no  differences  between  groups  in  treatment 
outcome.  There  was  even  no  trend  of  a  superior  treatment  outcome  for  the  VR-motion  compared  to 
the  VR-no-motion  group. 

Conclusions  and  novelty:  The  present  study  helps  to  better  understand  VR  exposure  treatment  and 
gives  hints  for  future  research  to  evaluate  the  treatment  process.  Based  on  our  results  it  may  be 
speculated  that  treatment  outcome  is  more  related  to  the  amount  of  habituation  during  exposure 
than  to  the  strength  of  the  initial  fear  response. 
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INTRODUCTION 

Many  studies  confirm  the  efficacy  of  Virtual 
Reality  (VR)  exposure  for  the  treatment  of  dif¬ 
ferent  psychiatric  disorders,  especially  for  spe¬ 
cific  phobia.  The  efficacy  of  VR  exposure 
for  the  treatment  of  fear  of  flying  was  con¬ 
firmed  in  more  than  8  controlled  clinical  trials.'*'^ 


185 


Annual  Review  of  CyberTherapy  and  Telemedicine 


Important  goals  of  future  studies  are  the 
evaluation  of  underlying  treatment  processes 
and  the  identification  of  crucial  treatment  com¬ 
ponents,  both  in  order  to  optimize  the  treat¬ 
ment.  VR  exposure  is  an  especially  valuable 
tool  to  achieve  this  since  it  allows  on  the  one 
hand  to  control  exposure  settings  and  on  the 
other  hand  to  create  a  wide  range  of  experi¬ 
mental  manipulations. 

For  exposure  therapy,  a  widely  accepted  theo¬ 
retical  assumption  is  that  the  fear  network  has 
to  be  activated  before  it  can  be  changed.  Foa 
and  Kozak®  further  assumed  that  this  activa¬ 
tion  has  to  be  reflected  in  initial  subjective  as 
well  as  physiological  fear  responses.  Following 
this  initial  activation,  an  effective  exposure 
treatment  should  lead  to  a  habituation  of  acti¬ 
vation  both  within  and  between  repeated  expo¬ 
sures.®  In  line  with  this  assumption,  an  asso¬ 
ciation  between  strong  initial  heart  rate  reac¬ 
tions  during  exposure  and  superior  treatment 
outcome  was  repeatedly  found. 

Network  theories  also  predict  that  effective  VR 
treatments  must  be  able  to  Induce  substantial 
fear  reactions.  In  flight  phobics,  such  substan¬ 
tial  fear  reactions  have  been  observed  in  VRs 
incorporating  visual,  acoustic  and  motion 
simulation^  or  visual,  acoustic  and  vibration 
simulation.^®  Variables  that  were  discussed  to 
moderate  the  amount  of  fear  elicited  by  VR 
exposure  are  the  presence  in  VR^^'^^  or  the 
feedback  of  physiological  activation  during 
VR.^  An  additional  important  variable  modulat¬ 
ing  subjective  and  physiological  fear  reactions 
within  VR  exposure  might  be  the  applied  simu¬ 
lation  modalities. 

The  present  study  was  designed  to  examine 
the  effect  of  motion  simulation  on  subjective 
and  physiological  fear  reactions  as  part  of  a 
VR  exposure  that  also  included  visual  and 
acoustic  stimuli.  We  considered  this  important 
since  motion  simulation,  on  the  one  hand, 
might  be  important  to  trigger  fear  during  virtual 
flights  but,  on  the  other  hand,  is  rather  expen¬ 
sive  to  add  because  it  needs  additional  hard¬ 
ware  (motion  chair).  Our  hypotheses  were  that 
motion  simulation  results  in  stronger  initial 
subjective  and  physiological  fear  reactions, 
and  as  a  consequence  is  associated  with 
stronger  habituations  of  fear  responses  within 


VR  flights  and  between  VR  flights.  Based  on 
earlier  results'*  we  specifically  expected  heart 
rate  to  habituate  mostly  between  flights  and 
skin  conductance  levels  to  habituate  especially 
strong  within  flights.  Furthermore,  we  assumed 
that  motion  simulation  enhances  treatment 
efficacy  as  measured  in  the  fear  of  flying  scale 
(FFS). 

METHOD 

Participants 

Participants  were  recruited  through  newspaper 
articles.  Respondents  completed  a  mailed 
questionnaire  package  that  included  a  socio¬ 
demographic  and  a  medical  history  question¬ 
naire,  and  several  psychometric  instruments 
which  assessed  fear  of  flying,  trait  anxiety,  and 
psychopathology.  For  a  detailed  description  of 
the  selection  procedure,  the  inclusion  and  ex¬ 
clusion  criteria,  the  complete  treatment  pro¬ 
gram  and  characteristics  of  the  participants 
see  MQhlberger  et  al.*®  Written  informed  con¬ 
sent  was  obtained  from  all  study  participants. 
The  presented  analyses  are  based  on  25  par¬ 
ticipants  with  fear  of  flying  with  complete  proc¬ 
ess  data:  All  participants  received  all  treatment 
components  including  written  information 
about  fear  of  flying  and  how  to  cope  with  it 
(information  booklet),  one  hour  of  cognitive 
therapy  (CT),  and  VR  exposure  (four  flights). 
However,  VR  exposure  included  for  twelve 
participants  motion  simulation  (VR-motion 
group)  while  thirteen  participants  (VR-no- 
motion  group)  received  VR  exposure  without 
motion  simulation.  For  the  presented  evalua¬ 
tion  of  treatment  outcome  missing  follow-up 
data  of  two  participants  of  the  no-motion  group 
was  completed  by  the  group  mean  of  all  par¬ 
ticipants  who  did  not  fly  during  follow-up  pe¬ 
riod. 

Apparatus 

Virtual  Reality  (VR):  The  VR  environment  in¬ 
cluded  visual,  acoustic,  and  partly  motion 
simulations  of  a  commercial  flight.  VR  flights 
lasted  18  min  and  included  all  important  flight 
phases  (i.e.,  take-off,  ascending,  reaching 
flight  altitude,  two  phases  of  air  turbulence, 
descending,  and  landing). 


186 


Muhlberger  et  al. 


The  visual  cues  were  presented  by  a  head- 
mounted  display  (HMD;  V6,  Virtual  Research 
Corporation).  The  head  position  was  moni¬ 
tored  with  an  electro-magnetic  tracking  device 
{Fast  Track,  Polhemus  Corporation)  in  order  to 
adapt  the  field  of  view  to  head  movements.  A 
Reality  Engine  ii  Computer  (Silicon  Graphics 
Corporation)  with  two  processing  units  pro¬ 
vided  real-time  texture  mapping  and  imple¬ 
mented  the  real-time  rendering  of  the  environ¬ 
ment  with  respect  to  the  head  position.  A  high 
amount  of  details  and  a  frame-rate  of  at  least 
12  frames  per  second  were  achieved  by  re¬ 
nouncing  stereoscopic  rendering. 

The  three-dimensional  environment  consisted 
of  the  inside  of  an  airplane  (Boeing  737)  and 
the  view  through  an  airplane  window 
(somewhat  enlarged  compared  to  reality).  The 
participant's  position  was  a  passenger  com¬ 
partment  at  the  left  window  row  in  the  middle 
of  the  plane.  The  seat  at  the  right  of  the  partici¬ 
pant  was  taken  by  a  virtual  passenger.  The 
participant  was  able  to  look  out  of  the  plane 
through  the  window  to  the  left.  The  outside 
environment  changed  dependent  on  the  flight 
phase  (e.  g.,  parking  position,  runway,  airport- 
tower,  several  airport-buildings  at  take-off  or 
landing,  fields  or  clouds  while  flying). 

Sounds  were  presented  by  an  Indigo  Com¬ 
puter  (Silicon  Graphics  Corporation)  via  ear¬ 
phone  integrated  in  the  HMD.  The  audio  cues 
were  original  airplane  sounds  and  flight  an¬ 
nouncements,  both  recorded  and  stored  as 
digital  audio  files. 

A  motion  base  with  6  degrees  of  freedom 
(Symtech  Corporation)  was  used  to  manipu¬ 
late  the  body  position  and  to  simulate  motions. 
Simulations  of  speed  accelerations,  speed 
deceleration  (during  take-off  or  landing),  and 
air  turbulence  (during  the  flight)  were  imple¬ 
mented.  The  motion  data  mirrored  a  flight  that 
was  performed  by  a  real  pilot  in  a  professional 
flight  simulator.  A  special  program  ("Wash¬ 
out")  developed  by  the  Fraunhofer-lnstitute  for 
Graphical  Data  Processing  (IGD)  in  Darmstadt 
was  used  to  transform  the  data  for  motion 
base  use  and  to  ensure  an  optimal  and  realis¬ 
tic  perception  of  speed  acceleration  or  speed 
deceleration. 


Relevant  psychometric  measures  (for  a  com¬ 
plete  list  see  Muhlberger  et  al.,  2003). 

Subjective  Units  of  Discomfort  (SUDs)^'^  ranging 
from  0  (no  discomfort)  to  100  (panic-like  dis¬ 
comfort)  were  used  to  assess  fear  responses. 
Participants  were  trained  to  use  this  scale,  and 
ratings  were  requested  at  eight  assessment 
points  per  flight  (before  the  start,  directiy  after 
the  start,  during  quite  flight,  during  two  phases 
of  turbulences,  once  again  during  quite  flight, 
after  landing  and  reaching  the  final  parking  po¬ 
sition)  via  earphone  instruction. 

The  Fear  of  Flying  Scale  consists  of  21 

items  describing  situations  representative  for  air 
travel  (e.g.,  "The  plane  accelerates  down  the 
runway  and  lifts  off  the  ground").  The  fear  elic¬ 
ited  by  the  described  situations  has  to  be  rated 
on  5-point  scales  (not  a  all  (0)  to  very  much 
(4)).  A  mean  score  was  calculated.  Cronbach's 
a  and  3  month  retest  reliability  of  the  original 
FFS  were  .94  and  .86,  respectively.^®  We  con¬ 
firmed  these  indices  for  our  German  translation 
in  two  studies:^®  Cronbach's  a  were  .90  (N=120) 
and  .98  (N=257):  3  month  retest  reliabilities 
were  .83  (N=  37)  and  .87  (N=43). 

PROCEDURE 

For  the  evaluation  of  treatment  outcome  par¬ 
ticipants  completed  the  FFS  before  and  after 
the  VR  exposure  treatment  and  6  month  later. 
For  a  detailed  description  of  the  complete 
treatment  and  the  details  of  the  outcome  as¬ 
sessment  see  Muhlberger  et  al.^® 

Participants  of  both  VR  groups  were  accompa¬ 
nied  to  the  VR  simulator  after  the  same  cogni¬ 
tive  preparation.  The  VR  devices  were  then 
explained.  Participants  sat  down  on  a  cush¬ 
ioned  seat  mounted  on  a  motion  platform  and 
the  HMD  was  put  on.  Before  starting  the  VR 
flights,  participants  were  instructed  not  to  sup¬ 
press  their  fear  during  the  VR  flights,  but  to 
experience  the  fear  and  its  reduction  with  on¬ 
going  exposure.  Participants  completed  four 
successive  VR  flights  of  18  min  duration  each; 
the  VR-motion  group  with  motion  (including 
vibration)  simulation,  the  VR-non-motion  group 
without  motion  (and  without  vibration)  simula¬ 
tion.  A  break  of  5  to  10  minutes  separated  the 
second  from  the  third  flight. 
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Physiological  data  acquisition  and  data  reduc¬ 
tion 

Electrocardiogram  (ECG)  and  skin  conductance 
level  (SCL)  were  registered  continuously  with  a 
sampiing  rate  of  384  Hz  with  a  Vitaport  I  system 
(Becker  Inc.).  Skin  conductance  was  recorded 
from  two  electrodes  placed  on  the  medial  pha¬ 
langes  of  the  second  and  third  finger  of  the  non¬ 
dominant  hand.  The  ECG  was  converted  on¬ 
line  to  heart  rate  (HR)  by  an  integrated  R-wave 
detection  algorithm.  Phases  of  60  s  duration 
corresponding  to  SLID  ratings  were  extracted 
and  means  for  each  phase  and  each  flight  were 
calculated.  Then,  differences  to  the  pre-flights 
baseline  measures  were  computed.  Physiologi- 
cai  signals  were  pre-analyzed  offline  with  the 
BrainVision  Anaiyser  Software  of  BrainProducts 
Inc. 

Statistical  data  analysis 

SUD,  HR  and  skin  conductance  were  ana¬ 
lyzed  with  mixed  ANOVAs  with  the  between 
factor  Motion  Group  (with  or  without)  and  the 
within  factors  Flight  (one  to  four)  and  Phase  (8 
assessment  points).  Treatment  outcome  was 
analyzed  with  mixed  ANOVA  with  the  between 
factor  Motion  Group  (with  or  without)  and  the 
within  factor  Time  (pre,  post,  foiiow-up).  If  ap¬ 
propriate,  Greenhouse-Geisser  corrections  of 
degrees  of  freedom  (df)  were  applied.  Signifi¬ 
cant  effects  were  foilowed  up  with  specific 
ANOVAs  or  pianed  contrasts. 

RESULTS 

Treatment  process 

Fear  reports  (SUDs):  Figure  1  depicts  the 
mean  fear  reports  (SUDs)  assessed  during  the 
four  VR  exposure  flights  separated  for  the 
groups  with  (VR  motion)  and  without  motion 
simulation  (VR  no-motion).  Strongest  fear  re¬ 
ports  were  obseryed  during  the  turbulence 
phases  of  the  first  VR  flight  with  a  mean  SUD 
above  40  in  the  VR  motion  group.  A  decrease 
of  fear  within  VR  exposure  flights  and  across 
repeated  VR  flights  is  clearly  visible. 

The  overall  analysis  revealed  significant  main 
effects  of  Flight  (F(3, 69)=24.7,  p<.001,  0° 


=0.52),  Phase  (F(7,i6i)=12.0,  p<.001,  □^=0.34), 
and  Motion  Group  (F(i,  23)=5.5,  p=.028,  0° 
=0.19),  and  a  significant  Motion  Group  by 
Phase  (F(7  161)=  4.6,  p=.005,  □°=0.17)  interac¬ 
tion.  Contrasts  revealed  a  significant  habitua¬ 
tion  from  flight  to  flight  {F^^23)=  10.1;  32.5; 
32.1,  p=.004;  <.001;  <.001,  □°=0.31;  0.59; 
0.58X  The  main  effect  of  Phase  can  be  traced 
back  to  enhanced  fear  during  the  start  (F(i  32) 
=17.3,  p<.001,  □°=0.43)  and  the  first  flight 
phase  (F(i,23)=17.3,  p<.001,  □“=  0.17),  and 
reduced  fear  after  landing  (F(i,23)=17.4,  p<.001, 
□'^=0.43),  compared  to  the  fear  before  the 
start  of  the  flight.  The  motion  group  displayed 
overall  enhanced  fear  responses,  which  were 
-  as  the  interaction  Motion  Group  by  Phase 
indicated  -  stronger  as  the  non-motion  group 
responses’  during  the  first  flight  phase  (F(i,23) 
=5.9,  p=.024,  □“=0.20)  and  both  turbuience 
phases  (F(i23)=4.1;  p=.055,  0^=0. 15;  and  F 
,i,23)=6.5,  p=.018,  ^^=0.22). 

Heart  rate:  The  analysis  returned  significant 
main  effects  of  Fiight  (F(3,63)=9.2,  p=.001, 
=0.30)  and  Phase  (F(3, 63)=3.3,  p=.023,  0° 
=0.13),  and  significant  Phase  by  Motion  Group 
(F(7,  i47)=4.3,  p=.006,  □’^=0.17)  and  Fiight  by 
Phase  (F(2i,441)=6.2,  p<.001,  □’^=0.23)  interac¬ 
tions.  Further  anaiyses  within  groups  returned 
no  significant  effects  for  the  VR  no-motion 
group.  For  the  VR  Motion  group,  the  main  ef¬ 
fects  Flight  (F(3,  30)=8.9,  p=,005,  □'^=0.47)  and 
Phase  (F(7,  70)=5.4,  p=.006,  □■^=0.35)  and  the 
interaction  Flight  by  Phase  (F(2i,  2io)=5.1, 
p<.001,  ^^=0.34)  were  significant.  These  ef¬ 
fects  are  based  on  the  strong  initial  heart  rate 
reaction  during  the  start  phase  of  flight  one,  a 
smaller  one  during  flight  two  (see  Figure  2), 
and  no  further  reactions  during  flight  tree  and 
four  (not  depicted). 

Skin  conductance  level:  The  analysis  yielded  a 
significant  main  effect  of  Phase  (F(7,i54)=4.8, 
p=.007,  □°=0.18)  and  significant  Flight  by  Mo¬ 
tion  Group  (F(3,66)=5.9,  p=.005,  □°=0.21)  and 
Phase  by  Flight  by  Motion  Group  (F(2i,462)=4.5, 
p=.005,  □°=0.17)  interactions.  A  separate 
ANOVA  for  the  VR  motion  group  revealed  a 
significant  main  effect  of  Flight  (F(3,3Q)=5.5, 
p=.017,G°=0.36).  Contrasts  returned  signifi¬ 
cant  habituations  between  the  second  and  the 
third  flight  (F(i,io)=5.9,  p=.035,dP=0.37)  and 
between  the  third  and  the  fourth  flight  (F(i,io) 
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before  take  off  flight  1  turb  1  turb  2  flight  2  landing  standing 


Figure  1.  SUD  ratings  for  the  first  two  (top)  and  the  last  two  (bottom)  VR  flights  of  the  Motion 
(N=12)  and  the  No-motion  (N=13)  groups.  Flights  have  eight  phases  including  two  phases  with 
turbulences  (turb) 
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Figure  2.  HR  responses  (top)  during  the  first  two  VR  flights  of  the  Motion  and  the  No-motion  groups 
(N  motion  =1 1 ,  N  no  motion  =  12)  and  skin  conductance  level  responses  (bottom)  during  the  first  two  VR 
flights  of  the  Motion  and  the  No-motion  groups  N  motion  =11.  N  no  motion  =  13);  Flights  have  eight 
phases  including  two  phases  with  turbulences  (turb). 
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=8.6,  p=.015,  □■=0.46)  (not  depicted).  The 
ANOVA  for  the  VR  no-motion  group  resulted  in 
a  significant  main  effect  of  Phase  (F(7,84)=3.6, 
p=.041;  □°=0.23)  and  a  significant  Phase  by 
Flight  (F(2i,252)=4.0,  p=.028;  □°=0.25)  interac¬ 
tion.  As  depicted  in  Rgure  2,  skin  conductance 
level  of  the  VR  no-motion  group  habituated 
strongly  during  the  first  flight  and  remained 
constant  during  the  second  flight. 

Treatment  outcome  (FFS) 

Within  the  VR-motion  group,  the  mean  FFS 
score  dropped  from  2.7  pre  treatment  (sd  = 
0.4)  to  1.9  post  treatment  (sd  =  0.4)  and  2.1 
six  months  later  (sd  =  0.7).  The  corresponding 
scores  for  the  VR-no-motion  group  were  2.5  at 
pre  treatment,  (sd  =  0.5),  1.7  post  treatment 
(sd  =  0.8)  and  1 .7  at  follow-up  (sd  =  0.  9). 
Significant  Time  main  effects  (F(2,46)=18.6, 
p<.001,  □^=0.45)  revealed  an  overall  treat¬ 
ment  efficacy.  The  Motion  Group  by  Time  in¬ 
teraction  was  not  significant  (F(2,46)=0.5, 
p=.489,  0°  =0.02).  Contrasts  revealed  that 
fear  of  flying  was  effectively  reduced  from  pre 
to  post  treatment  (F(i,23)=39.7,  p<.001,  0° 
=0.63)  and  from  pre-treatment  to  follow-up  (F 
(i,23)=21.6,  p<.001,  □^=0.49). 

DISCUSSION 

Results  support  prior  investigations  that  found 
substantial  subjective  fear  reactions  of  flight 
phobics  during  virtual  reality  (VR)  flights.  How¬ 
ever,  the  VR  with  motion  simulation  induced 
substantially  stronger  subjective  fear  over  all 
flights  and  all  flight  phases  than  the  VR  without 
motion  simulation.  Especially  pronounced  dif¬ 
ferences  between  the  motion  and  the  no¬ 
motion  groups  were  found  during  the  first  flight 
before  and  during  phases  with  intensive  mo¬ 
tion  simulation,  namely  the  phases  with  simu¬ 
lation  of  air  turbulences.  SUD  ratings  de¬ 
creased  immediately  after  the  start  phase  in 
the  group  without  motion  simulation,  while  fear 
remained  enhanced  in  the  VR  motion  group 
until  the  phases  with  turbulences  were  com¬ 
pleted. 

Both  treatment  groups  displayed  a  habituation 
of  subjective  fear  responses  from  flight  to 
flight.  However,  the  expected  association  be¬ 
tween  strong  initial  fear  reactions  and  strong 
habituation  effects  was  not  confirmed.  Both 


groups  did  not  differ  in  the  strength  of  habitua¬ 
tion.  As  a  consequence,  fear  responses  at  the 
end  of  the  exposure  of  the  VR-motion  group 
were  still  higher  than  the  responses  of  the  VR- 
no-motion  group,  although  both  groups  had 
widely  diminished  subjective  fear  responses. 

Fear  associated  heart  rate  responses  could 
only  be  detected  in  the  VR-motion  group.  Sig¬ 
nificant  HR  accelerations  were  observed  dur¬ 
ing  the  first  flight’s  take  off  phase  and  -  to  a 
weaker  extend  -  during  the  second  flight’s 
take  off.  During  later  flights,  HR  was  habitu¬ 
ated,  and  further  significant  reactions  did  not 
occur.  The  VR  group  without  motion  simulation 
showed  no  significant  HR  reactions  at  all. 

Skin  conductance  level  of  the  VR  motion 
group  remained  enhanced  during  the  first  and 
second  flights  and  habituated  only  afterwards. 
In  contrast,  the  VR  no-motion  group  displayed 
enhanced  SCL  responses  only  during  the  take 
off  phase  of  the  first  flight;  SCL  responses 
were  habituated  during  later  flight  phases  and 
later  flights. 

Overall,  subjective  fear  ratings  as  well  as  skin 
conductance  responses  confirmed  substantial 
fear  of  both  groups  during  VR  exposure.  How¬ 
ever,  these  responses  were  substantially 
stronger  and  habituated  slower  in  the  VR  mo¬ 
tion  group  compared  to  the  VR  no-motion 
group.  Nevertheless  and  in  contrast  to  network 
theories  -  which  suggest  that  stronger  activa¬ 
tion  of  fear  networks  should  result  in  an  en¬ 
hanced  treatment  outcome  -  we  found  no  dif¬ 
ferences  between  groups  in  treatment  out¬ 
come.  There  was  even  no  trend  of  a  superior 
treatment  outcome  for  the  VR-motion  com¬ 
pared  to  the  VR-no-motion  group.  Both  groups 
also  showed  comparable  habituation  effects, 
although  the  habituation  within  the  VR-motion 
group  took  place  at  a  higher  level  (subjective 
fear)  or  had  a  later  onset  (skin  conductance). 
Although  these  results  are  limited  due  to  the 
small  sample  sizes,  the  present  study  helps  to 
better  understand  VR  exposure  treatment  and 
gives  hints  for  future  research  to  evaluate  the 
treatment  process.  Based  on  our  results  it  may 
be  speculated  that  treatment  outcome  is  more 
related  to  the  amount  of  habituation  during 
exposure  than  to  the  strength  of  the  initial  fear 
response.  Furthermore,  since  activation  and 
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habituation  are  highly  related  In  exposure 
therapies/®  the  manipulation  of  distinct  stimuli 
properties  within  VR  exposure  seems  to  be  a 
valuable  tool  to  clarify  the  active  components 
of  exposure  treatments. 
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Abstract:  Patients  who  have  had  a  stroke  constitute  a  large  population  with  significant  needs  for  re¬ 
habilitation.  Even  after  long,  intensive  and  costly  rehabilitation,  these  patients  still  suffer  from  many 
impairments  leading  to  severe  restrictions  in  participation  in  every  day  life.  This  is  a  result  of  limited 
functional  recovery  of  the  upper  extremity  in  addition  to  cognitive  and  meta-cognitive  deficits.  It  is 
also  due  to  insufficient  training  of  lADL  functioning  during  rehabilitation,  primarily  since  such  training 
is  both  time  consuming  and  technically  difficult  to  implement.  The  use  of  VR  for  rehabilitation  of  indi¬ 
viduals  with  neurological  impairments  appears  to  be  promising  and  its  feasibility  and  effectiveness 
have  been  demonstrated  in  a  number  of  different  studies.  However,  the  VR-based  interventions  that 
have  been  developed  and  reported  in  the  literature  used  mainly  desktop  platforms,  and  the  majority 
of  them  did  not  include  the  stroke  population.  The  aim  of  this  paper  is  to  present  the  development  of 
a  virtual  environment  using  a  video  capture  VR  system,  which  provides  an  opportunity  to  carry  out 
therapeutic  intervention  in  a  functional,  natural  and  a  motivating  way.  The  results  of  a  pilot  study  on 
patients  with  stroke  highlighted  its  many  potential  advantages.  These  results  as  well  as  future  direc¬ 
tions  will  be  discussed. 


INTRODUCTION 

Stroke  is  considered  to  be  the  third  leading 
cause  of  death  and  the  most  important  cause  of 
severe  disability  in  old  age.^'^  Almost  half  of  the 
patients  suffering  from  stroke  retain  substantial 
disability^  affecting  their  performance  of  activi¬ 
ties  of  daily  living.  The  main  symptom  following 
stroke  is  paresis  or  paralysis  to  half  of  the  body, 
contra  lateral  to  the  side  of  brain  lesion.  Addi¬ 
tional  symptoms  include  sensory,  perceptual, 
cognitive  and  meta-cognitive  deficits,  which  re¬ 
sult  in  a  decreased  ability  to  perform  activities 
of  daily  living.'*  Patients,  who  demonstrate  diffi¬ 
culties  functioning  in  activities  of  daily  living  of¬ 
ten  suffer  from  deficits  in  executive  functions 
despite  being  cognitively  intact.®®  These  pa¬ 
tients  present  problems  such  as  starting  and 
stopping  activities,  an  inability  to  shift  mental 
processes  and  behavioral  tasks,  planning  and 
monitoring  deficits  as  well  as  keeping  a  goal  in 
mind  throughout  a  task.®  The  assessment  and 
rehabilitation  of  executive  functions  under  non¬ 
natural  conditions  such  as  a  research  labora¬ 
tory  or  clinical  facility  is  still  very  unsatisfactory, 
and  the  evaluation  of  real  life  tasks,  which  has 
the  advantage  of  giving  a  more  accurate  appre¬ 


ciation  of  the  patient’s  deficits,  appears  to  be 
time  consuming  and  not  suitable  for  the  typical 
clinical  setting.' 

The  functional  recovery  of  the  upper  extremity 
after  stroke  is  considered  to  be  exceptional  in 
most  cases;  the  arm  does  not  usually  become 
functional  even  after  intensive  and  long¬ 
standing  therapy.®  Most  of  patients  quickly  learn 
to  cope  by  using  only  the  unaffected  arm,  and, 
indeed,  prefer  to  avoid  failure  by  not  using  the 
affected  arm  at  all.  This  phenomenon  is  known 
as  "learned  non-use."®  Recent  research,  using 
"Constraint  Induced  Movement  Ther¬ 
apy"  (CIMT)  has  been  shown  to  be  very  effec¬ 
tive^®  due  to  the  fact  that  patients  use  their 
hands  in  a  functional  manner  (and  not  as  rou¬ 
tine  exercises)  many  times  a  day  (high  repeti¬ 
tion)  which  endeavors  to  stimulate  healing  proc¬ 
ess  associated  with  plasticity  of  the  brain.” 

The  objective  of  rehabilitation  is  to  decrease  the 
various  motor,  cognitive  and  meta-cognitive 
deficits  described  above,  in  order  to  promote 
the  functional  ability  of  basic  and  instrumental 
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activities  daily  living  (BADL  and  lADL).  To 
date,  most  clinical  settings  rely  on  the  use  of 
non-functional,  routine  exercises  aimed  at  de¬ 
creasing  impairment,  despite  the  fact  that  it 
would  appear  to  be  much  more  relevant  and 
motivating  for  patients  to  exercise  while  en¬ 
gaged  in  meaningful,  purposeful  tasks  that  are 
related  to  real  life  interests  and  activities. 
Unfortunately  there  are  limited  opportunities  for 
implementing  purposeful  activities  within  tradi¬ 
tional  clinical  settings  via  conventional  occupa¬ 
tional  therapy.  Therefore  many  stroke  patients 
do  not  return  to  their  premorbid  functional  level, 
especially  in  the  more  complex  instrumental 
activities  of  daily  living. 

Virtual  Reality  (VR)  has  the  potential  to  be  used 
as  a  novel  modality  in  rehabilitation  assessment 
and  intervention  due  to  its  well  known  attrib¬ 
utes.^®'^®  To  date,  a  few  studies  have  examined 
the  use  of  VR  with  a  stroke  population,  and  in 
most  of  these  studies  very  small  numbers  of 
patients  participated.  Kizony  et  al.  assessed 
the  sense  of  presence  and  whole  body  perform¬ 
ance  in  virtual  games  of  patients  with  stroke.^^ 
Piron  et  al  used  VR  to  train  upper  limb  reaching 
movements^®  and  Broeren  et  al.  developed  a 
VR  haptic  device  for  the  assessment  and  train¬ 
ing  of  motor  coordination^®  and  Merians  et  al. 
used  a  force  feedback  glove  to  improve  range 
of  motion,  speed  and  strength  of  hand  move¬ 
ment.^®  Safe  street  crossing  which  is  a  major 
concern  was  trained  with  stroke  patients  who 
suffer  unilateral  neglect  using  a  desktop  VR 
system  and  found  to  be  useful.^^ 

In  order  to  encourage  activation  of  a  specific 
limb,  the  GX  system  may  be  operated  while 
users  wear  a  red  glove  on  one  or  both  hands.  If 
the  aim  is  to  improve  a  user’s  weak  upper  ex¬ 
tremity,  the  red  glove  will  be  worn  only  on  the 
impaired  hand  thereby  restricting  responses 
within  the  virtual  environment  to  that  specific 
body  part.^° 

To  summarize,  VR  generally,  and  the  GX  VR 
system  especially,  appear  to  have  considerable 
potential  for  assessing  and  treating  executive 
functions  and  the  upper  extremity  motor  func¬ 
tion  of  patients  with  stroke.  The  importance  in 
developing  additional  intervention  tools  that  are 
motivating  and  challenging  for  patients  with 
stroke  and  other  neurological  disorders  who  are 


in  need  of  lengthy  rehabilitation  and,  in  particu¬ 
lar,  opportunities  to  practice  instrumental  activi¬ 
ties  of  daiiy  living  led  us  to  develop  the  Virtual 
Mall  using  the  GX-VR  system.  The  Virtual  Mall 
(VMall)  was  developed  and  designed  to  facili¬ 
tate  assessment  and  treatment  whilst  the  pa¬ 
tient  is  engaged  in  a  complex,  everyday  task  of 
shopping.  The  aims  of  this  paper  are  (1)  to  de¬ 
scribe  the  development  of  a  functional  virtual 
environment  and  it’s  usability  for  stroke  patients 
(2)  to  present  the  results  of  a  pilot  study  with 
eight  stroke  patients. 

DEVELOPMENT  OF  THE  VIRTUAL  MALL 
(VMALL) 

The  VMall  aims  to  simulate  a  real  shopping 
mall  consisting  of  different  stores  in  order  to 
provide  patients  a  functional  and  ecological 
valid  environment  which  can  be  used  to  train 
motor,  cognitive,  and  metacognitive  skills  as 
well  as  a  shopping  task  itself. 

The  VMall  was  developed  primarily  for  use  with 
neurological  patients  who  suffer  from  motor, 
cognitive  and/or  metacognitive  deficits.  The  first 
population  tested  was  adults  and  elderly  pa¬ 
tients  who  suffer  from  stroke  and  whose  barri¬ 
ers  for  independence  in  daily  living  were  de¬ 
scribed  above.  Thus  it  was  important  for  the 
VMall  to  facilitate  active  movement  and  execu¬ 
tive  functions  while  engaged  in  the  shopping 
task.  In  addition,  the  consideration  of  rehabili¬ 
tation  principles  including  flexibility  of  the  envi¬ 
ronment  and  the  ability  to  adapt  the  task  to  suit 
the  patient’s  level  of  ability  was  also  Important. 

To  date,  the  VMall  is  a  large  supermarket  con¬ 
sisting  of  many  different  aisles.  Photographs  of 
real  grocery  items  were  taken  with  a  digital 
camera  and  rendered  via  3-D  graphic  software. 
These  graphic  files  were  then  integrated  into 
the  VMall  virtual  environment.  Each  aisle  is 
composed  of  a  maximum  of  60  products  lo¬ 
cated  on  shelves,  and  sorted  into  different  cate¬ 
gories  such  as  baking  goods,  cleaning  items, 
and  stationary  products.  The  names  of  the 
aisles  appear  on  a  sign  which  also  has  an  im¬ 
age  of  the  type  of  products  located  in  that  aisle 
(see  figure  1 ,  right  panel). 

The  products  may  be  selected  and  sorted  by  a 
therapist  thereby  providing  flexibility  with  regard 
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Figure  1:  Screen  shots  of  the  VMall  showing  a  user  selecting  a  shopping  aisle  (left  panel),  a  food 
item  (middle  panel)  and  verifying  the  contents  of  the  shopping  cart  (right  panel). 


to  the  number,  type  and  location  of  items  within 
each  aisle.  Four  copies  of  each  product,  one 
behind  the  other,  are  located  on  the  shelves 
such  that  following  the  purchase  of  one  item, 
another  three  may  be  seen  (and  purchased). 
Once  all  four  items  are  bought,  the  shelf  is  bare 
in  that  location  (see  figure  1 ,  middle  panel). 

The  names  of  each  product  have  been  re¬ 
corded.  Thus,  in  addition  to  the  visual  display 
of  the  grocery  items,  the  name  of  the  product  is 
heard  whenever  it  is  touched.  The  item  is  also 
enlarged  when  it  is  touched.  This  augmented 
visual  and  auditory  feedback  has  been  added 
to  ensure  that  the  user  has  correctly  identified 
the  desired  product. 

Several  additional  features  have  been  included 
to  enhance  the  user’s  sense  of  being  present 
within  a  supermarket  environment.  Background 
“shopping  mall”  type  music  is  played  throughout 
the  shopping  experience.  Typical  announce¬ 
ments  of  special  sales  are  also  heard  occasion¬ 
ally  during  the  shopping  task.  Such  announce¬ 
ments  may  be  recorded  easily  by  a  therapist 
and  suited  to  a  particular  patient  or  shopping 
task. 

ACTIVATION  AND  OPERATION  OF  THE 
VMALL 

The  user  sees  himself  within  the  VMall  and  the 
navigation  is  done  virtually  touching  arrows, 
which  scrolls  the  screen  to  the  left  or  right  side. 
Entering  the  desired  aisles  is  done  by  touching 
the  sign  of  the  aisle  and  selecting  an  item  is 
done  by  touching  it. 


At  first,  the  users  has  to  select  the  desired  aisle 
in  accordance  with  the  agreed  upon  shopping 
list.  Once  an  aisle  is  entered,  the  user  faces 
shelves  stocked  with  different  products.  The 
products  may  be  “purchased”  by  touching  them 
with  either  hand  (provided  that  a  red  glove  is 
worn).  The  selected  product  then  appears  in  a 
shopping  cart  (which  may  be  viewed  at  any 
time  by  linking  to  a  second  display  screen)  (see 
figure,  right  panel).  Items,  acquired  by  mistake 
or  not  needed,  may  be  removed  from  the  cart, 
by  touching  them.  The  user  can  return  from  the 
shopping  cart  screen  to  the  aisle  screen  to  con¬ 
tinue  shopping,  and  from  each  aisle  the  user 
can  return  to  the  main  screen  to  choose  an¬ 
other  aisle.  Patients  may  be  given  different 
tasks  to  do,  including,  for  example,  the  selec¬ 
tion  of  a  specific  recipe,  compilation  of  the  req¬ 
uisite  shopping  list  (in  order  to  obtain  the  prod¬ 
ucts  needed  for  that  recipe),  and  the  purchase 
of  products  from  this  list. 

Output  measures  include  data  indicating  how 
well  the  user  accomplished  the  task  (e.g.,  how 
many  correct  items  were  selected,  time  to  ac¬ 
complish  the  task,  sequence  of  items  selected, 
the  number  of  times  the  user  checked  his  shop¬ 
ping  cart)  which  are  recorded  and  saved  auto¬ 
matically  thereby  providing  a  record  of  improve¬ 
ment  over  time.  We  are  currently  investigating 
the  possibility  of  recording  the  patient’s  limb 
and  trunk  kinematics  while  using  the  VMall. 

USABILITY  STUDY 

Following  the  initial  development  of  the  VMall, 
six  community  dwelling  individuals  with  stroke 
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at  a  chronic  stage  were  invited  to  experience 
the  environment.  The  aim  was  to  test  its  suit¬ 
ability  for  the  use  with  stroke  patients  and  to 
receive  user-centered  feedback  prior  to  com¬ 
pletion  of  the  environment. 

Three  males  and  three  females,  mean  age  60.3 
±9.5  (range  50-74  years),  who  sustained  a 
right  hemispheric  stroke  1 .5  to  5  years  prior  to 
testing,  experienced  the  VMall.  They  all  suf¬ 
fered  from  varying  degrees  of  weakness  in  their 
affected  upper  extremity  but  had  no  cognitive 
deficits.  Five  patients  were  able  to  walk  and 
one  was  independently  mobile  in  a  wheel  chair. 
Four  of  them  go  shopping  on  a  regular  basis. 

They  experienced  the  VMall  for  approximately 
ten  minutes  while  receiving  an  explanation 
about  how  to  operate  it.  They  then  were  given 
a  six  item  shopping  list  and  were  asked  to  shop 
for  the  items.  When  the  task  was  completed 
they  filled  in  a  feedback  questionnaire  (the  Sce¬ 
nario  Feedback  Questionnaire  (SFQ)^^  with 
several  additional  questions  related  to  VMall 
usability  (e.g..  To  what  extent  were  the  prod¬ 
ucts  clear?).  They  were  also  asked  to  rate  their 
level  of  perceived  exertion  (Borg’s  scale)^®.  A 
short  interview  was  done  at  the  end. 

Overall  the  patients  all  enjoyed  their  virtual 
shopping  experience  very  much  and  said  they 
would  very  much  like  to  use  it  again.  They  ail 
found  the  task  to  be  challenging  and  interest¬ 
ing,  and  commented  that  it  had  potential  to  use 
in  rehabilitation.  The  mean  SFQ  scores  was 
25.6  ±  3.4  points  (maximum  30  points)  and  the 
mean  rated  exertion  was  13  ±  1.2  points  which 
indicates  an  easy  level.  It  was  interesting  to 
observe  that  several  patients  decided  to  buy 
items  that  were  not  on  the  list  because  they 
needed  them  at  home!  Others  bought  products 
that  were  on  sale.  From  these  responses  we 
learn  that  the  task  appeared  to  be  relevant  and 
realistic  for  the  participants,  and  that  they  felt  a 
high  level  of  presence.  After  using  the  VMall, 
one  male  user  remarked:  “During  the  experi¬ 
ence  ones  uses  all  of  his  senses,  also  his 
imagination  and  brain.”  A  female  user  stated  “It 
forced  me  to  straighten  my  fingers  which  I 
never  do  spontaneously.”  A  second  female  user 
commented  “It  made  me  feel  as  I  was  capable.” 


During  the  usability  study  we  encountered  a 
few  technical  difficulties  with  the  system  which 
have  now  been  resolved.  The  requisite  red 
glove  was  difficult  to  put  on  the  weak  hand  on 
some  individuals,  and  it  was  therefore  replaced 
with  a  red  mitten.  One  72  year  old  patient  com¬ 
plained  that  the  products  were  difficult  to  iden¬ 
tify,  and  the  resolution  of  the  3D  images  has 
now  been  improved.  One  female  participant 
said  the  task  was  not  relevant  for  her  since  she 
never  went  shopping  prior  to  her  stroke,  and 
was  not  planning  to  do  so  in  the  future.  This 
point  is  crucial  since  relevance  Is  a  key  objec¬ 
tive  in  using  the  VMall  as  a  therapeutic  task. 
We  therefore  now  include  an  initial  relevancy 
question  prior  to  usage;  “Did  you  go  shopping 
prior  to  your  stroke?” 

PILOT  STUDY 

Following  these  improvements,  eight  additional 
patients  with  stroke  experienced  the  VMall. 
They  were  all  males  and  their  ages  ranged  from 
38-73  years  (mean  58.7±  13.3  years).  Most  of 
them  were  during  their  sub-acute  rehabilitation 
(3-7  months  post  stroke)  whilst  one  patient  was 
four  years  post  stroke.  Five  patients  had  sus¬ 
tained  a  right  hemispheric  stroke  which  caused 
weakness  to  their  left  side  of  the  body  and  the 
other  two  sustained  a  left  hemispheric  stroke. 
All  patients  used  to  shop  alone  or  with  a  spouse 
prior  to  the  stroke.  One  patient  (44  years  old) 
was  a  butcher  in  a  supermarket  prior  to  his 
stroke.  They  did  not  have  cognitive  deficits  (the 
mean  Mini  Mental  State  Examination  (MMSE) 
scores  ranged  from  26-30  points)  but  all  suf¬ 
fered  from  a  different  degree  of  motor  impair¬ 
ment  of  their  upper  extremity  (upper  extremity 
subtest  of  the  Fugl-Meyer  Motor  assessment 
(FMA)  scores  ranged  from  7-58  points,  mean 
35.6  ±  17.5  points). 

The  protocol  was  identical  except  for  the  inter¬ 
view  which  was  not  conducted  and  extra  details 
of  the  patient’s  impairments  were  collected. 
This  included  the  MMSE  which  was  used  as  a 
cognitive  screening  test  (maximum  score  30 
points)  and  FMA  (maximum  score  60  points)  to 
assess  the  affected  arm’s  motor  impairment. 

RESULTS 

The  SFQ  scores  ranged  from  19-28  points  with 
a  mean  of  22.6±  3.2  points.  For  the  first  ques- 
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tion  “To  what  extent  did  you  enjoy  yourself?” 
the  mean  score  was  4.1  ±  0.99  points 
(maximum  5  points).  The  perceived  exertion 
level  was  13.2  ±  3.8  points  (which  ranged  from 
8-17  points  on  Borg’s  scale)  and  no  side  effects 
were  reported.  From  these  results  it  can  be 
seen  that  overall  these  stroke  patients  who 
were  relatively  a  short  time  post  stroke  with  dif¬ 
ferent  degrees  of  motor  impairments  were  able 
to  virtually  shop  for  products  using  one  or  both 
of  their  hands.  They  too  enjoyed  the  task  and 
thought  it  to  be  relevant  for  their  rehabilitation. 


CONCLUSIONS  AND  FUTURE  DIRECTIONS 

The  VMall  seems  to  have  great  potential  for  the 
use  with  stroke  patients  during  different  rehabili¬ 
tation  stages  since  it  provides  an  interesting 
and  motivating  task  which  encourages  active 
movement  especially  of  the  weak  upper  extrem¬ 
ity  and  facilitates  the  use  of  executive  functions. 
Recently  a  protocol  for  intervention  of  stroke 
patients  who  suffer  from  motor  and/or  executive 
functions  deficits  was  developed  and  a  study 
assessing  the  treatment  efficacy  has  started. 
We  also  plan  to  use  the  VMall  with  different  pa¬ 
tient’s  population  such  as  adolescents  post 
stroke  and  patients  suffering  from  traumatic 
brain  injury. 
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Abstract  The  aim  of  the  project  is  to  apply  augmented-reality  technology  to  teach  motor  skills  to  pa¬ 
tients  suffering  from  stroke.  To  achieve  this  goal  the  project  adopts  an  innovative  approach  based  on 
the  use  of  so-called  “motor  imagery. "  Recent  studies  in  neuroscience  have  provided  converging  evi¬ 
dence  that  “imagining"  a  motor  action  involves  the  same  brain  areas  involved  in  performing  the  ac¬ 
tion.  This  supports  the  idea  -  already  exploited  by  sports  trainers  -  that  training  with  motor  imagery 
(mental  practice)  could  be  effective  in  learning  new  motor  skills  or  in  promoting  motor  recovery  after 
damage  to  the  central  nervous  system.  Previous  clinical  studies  have  shown  that  the  rehabilitation  of 
post-stroke  hemiplegic  patients  can  be  made  more  effective  by  combining  physical  practice  with 
mental  practice.  However,  for  many  patients  who  suffer  from  damage  to  the  central  nervous  system, 
mental  simulation  of  movements  can  be  difficult,  even  when  the  relevant  neural  circuitry  has  not  been 
injured.  Starting  from  these  premises,  we  have  designed  and  developed  an  augmented-reality  work¬ 
bench  (called  “VR  Mirror”)  to  help  post-stroke  hemiplegic  patients  evoking  motor  images.  First,  the 
movement  is  acquired  by  the  system  from  the  healthy  arm.  Second,  the  movement  is  being  mirrored 
and  displayed  so  that  the  patient  can  observe  and  see  as  if  the  impaired  arm  is  performing  the  move¬ 
ment.  Third,  the  patient  is  instructed  to  rehearse  in  his/her  imagination  the  movement  he/she  has  just 
obsen/ed.  Last,  the  patient  has  to  perform  the  movement  with  the  affected  arm.  In  this  article,  we  de¬ 
scribe  preliminary  results  of  a  pilot  clinical  study,  which  has  evaluated  the  feasibility  of  using  technoi- 
ogy-supported  mental  simulation  as  an  adjunct  in  the  rehabilitation  of  the  upper  limb  following  stroke. 
DESIGN:  Single-case  study.  SETTING:  Physical  Rehabilitation  Unit  of  Padua  Teaching  Hospital,  Pa¬ 
dua,  Italy.  SUBJECTS:  A  46-year-old  man  with  stable  motor  deficit  of  the  upper  right  limb  following 
stroke.  INTERVENTION:  The  patient  underwent  a  single-case  design,  with  four  weeks  intervention. 
The  intervention  consisted  of  3  practice  sessions  per  week  at  the  hospital  using  the  VR-Mirror,  in  ad¬ 
dition  to  usual  therapy  (Bobath  method).  This  intervention  was  followed  by  1 -month  home- 
rehabilitation  program  using  a  portable  device.  MAIN  OUTCOME  MEASURES:  The  patient  was 
evaluated  for  level  of  Impairment  and  disability.  Pretreatment  and  posttreatment  measures  included: 
the  upper-extremity  scale  of  the  Fugl-Meyer  Assessment  of  Sensorimotor  Impairment  and  the  Action 
Research  Arm  Test.  RESULTS:  The  patient  showed  improvement  in  upper  limb  score  as  measured 
by  the  two  scales.  CONCLUSIONS:  The  improvement  observed  in  the  patient  is  encouraging  and 
warrants  further  study. 


INTRODUCTION 

Hemiplegia  is  total  paralysis  of  the  arm,  leg,  and 
trunk  on  one  side  of  the  body.  The  paralysis  pre¬ 
sents  as  weakness  that  may  be  associated  with 
abnormal  muscle  tone  (e.g.,  rigidity  or  spastic¬ 
ity).  The  most  common  cause  is  stroke,  which 
occurs  when  a  rupture  or  blood  clot  reduces 
blood  flow  to  a  specific  area  of  the  brain,  killing 
cells  and  disrupting  the  abilities  or  functions  they 


control.  Hemiplegia  can  be  caused  by  damage 
to  a  variety  of  structures  including  primary  sen¬ 
sorimotor  areas,  supplementary  motor,  premo¬ 
tor,  and  parietal  cortices,  basal  ganglia  and/or 
the  thalamus.^  Traditional  rehabilitation  after 
stroke  focuses  on  passive  (non-specific)  move¬ 
ment  or  on  compensatory  training  of  the  non¬ 
paretic  arm.  However,  only  a  small  percentage 
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(5-52%)  of  patients  regain  functional  recovery  of 
the  upper  extremity.^  This  observation  may  be 
related  to  the  limited  effectiveness  of  current 
therapy  techniques  used  for  improving  upper 
limb  function,  and  the  very  small  percentage  of 
the  patient’s  day  actually  spent  in  upper  limb 
intervention.^  Clinical  research  has  demon¬ 
strated  that  intensive,  repetitive  practice  of  active 
functional  tasks  shows  more  positive  outcomes 
for  upper  limb  rehabilitation  (Langhorne  et  al, 
1996).  It  has  been  suggested  that  these 
changes  occur  by  modifying  neural  reorganiza¬ 
tion  of  the  cerebral  cortex.^  In  particular,  there  is 
evidence  that  the  reorganization  of  the  cortex 
may  be  stimulated  not  only  by  physical  practice 
of  the  task,  but  also  by  mental  practice  of  the 
task  without  any  accompanying  movement.'^ 

Mental  practice,  also  called  symbolic  rehearsal 
or  motor  rehearsal,  is  a  training  technique  in 
which  the  subjects  repeatedly  “rehearse”  a  motor 
act  in  working  memory,  without  producing  any 
overt  motor  output.®®  In  recent  years,  several 
clinical  studies  have  described  the  contribution 
of  motor  imagery  practice  for  improving  upper 
limb  motor  function  after  stroke.^'^^  From  a 
neuropsychological  point  of  view,  the  benefit  of 
mental  practice  would  be  to  repetitively  activate 
cerebral  and  cerebellar  sensorimotor  structures 
damaged  by  a  stroke,  thereby  engaging  com¬ 
pensatory  networks  to  promote  motor  rehabilita- 
tion.'*’^^  Recent  neuroimaging  experiments  using 
positron  emission  tomography  (PET)  and  func¬ 
tional  magnetic  resonance  imaging  (fMRI)  have 
shown  that  imagining  a  motor  act  is  a  cognitive 
task  that  engages  parts  of  the  executive  motor 
system:  in  particular  the  supplementary  motor 
area,  premotor  area,  superior  parietal  lobule, 
and  cerebellum.'*  Efferent  discharges  produced 
during  imagery  may  also  activate  descending 
motor  pathways.  Motor  imagery  increases  spinal 
reflex  excitability  at  a  level  only  slightly  weaker 
than  during  movement,  and  corticospinal  excit¬ 
ability  is  similar  during  imagery  and  movement.*^ 
However,  for  many  patients  with  damage  to  the 
central  nervous  system,  mental  simulation  of 
movements  can  be  difficult,  even  when  the  rele¬ 
vant  neural  circuitry  has  been  spared.*®  The  goal 
of  this  project  was  thus  to  develop  technology 
tools  to  assist  patients  in  creating  motor  imagery. 
In  the  following  section,  we  describe  a  pilot  clini¬ 
cal  study,  which  has  evaluated  the  feasibility  of 
using  technology  to  assist  patients  in  creating 


motor  imagery.  The  rehabilitation  protocol  con¬ 
sists  of  an  inpatient  and  an  outpatient  phase, 
combining  physical  and  mental  practice.  In  the 
inpatient  phase,  patient  was  trained  in  a  labora¬ 
tory  setting,  using  a  custom-made  augmented- 
reality  workbench  (VR-Mirror).  At  the  end  of  this 
phase,  the  patient  used  a  portable  display  device 
to  guide  mental  and  physical  practice  at  his 
home. 

METHODS 

Recruitment 

Upon  an  appeal  by  the  researcher  to  individuals 
who  had  been  discharged  from  the  rehabilitation 
setting,  VP  was  the  first  volunteer  who  satisfied 
inclusion  criteria  for  participation  in  the  pilot  ex¬ 
periment.  VP  was  a  46-year-old  man  hospital¬ 
ized  from  early  November  2003  until  end  of  Feb¬ 
ruary  2004  after  an  ischemic  stroke  with  right 
hemiplegia. 

Comparison  between  our  observations  at  initial 
screening  and  medical  records,  discharge  sum¬ 
maries,  therapist  observations,  and  physiatrist 
observations  suggested  that  the  patient’s  af¬ 
fected  limb  function  had  not  improved  since  the 
time  of  discharge  from  the  hospital.  Before  start¬ 
ing  the  intervention,  VP  signed  an  informed  con¬ 
sent  statement,  in  accordance  with  the  guide¬ 
lines  of  the  institutional  ethical  review  board,  out¬ 
lining  his  rights  as  a  subject. 

Neuropsychological  assessment 

VP  was  shown  to  have  normal  communication 
and  cognitive  skills,  as  measured  by  the  Folstein 
Mini-Mental  State  Examination with  his  score 
of  28/30  falling  within  the  normal  range.  The  Mini 
Mental  State  Examination  is  a  widely  used  stan¬ 
dardized  method  by  which  to  grade  cognitive 
mental  status.  It  assesses  orientation,  attention, 
immediate  and  short-term  recall,  language,  and 
the  ability  to  follow  simple  verbal  and  written 
commands.  The  patient’s  ability  to  image  men¬ 
tally  was  measured  using  the  Vividness  of  Visual 
Imagery  Questionnaire.  In  the  WIQ  patient  is 
asked  to  imagine  different  scenarios  and  rate  the 
vividness  of  the  images  generated.  The  re¬ 
sponses  to  all  the  questions  can  be  summed  to 
provide  an  overall  score.  VP  scored  75/80  on  the 
WIQ,  showing  good  visual  imagery  ability.  This 
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result  was  confirmed  by  the  positive  perform¬ 
ance  on  the  Shepard-Metzler  Mental  Rotation 
Test,  which  measures  the  ability  to  mentally  ro¬ 
tate  three-dimensional  objects.  The  Vividness 
of  Movement  Imagery  Questionnaire'^  (VMIQ) 
which  is  extensively  used  in  sports,  was  used  to 
test  VP's  ability  to  imagine  prior  to  his  engage¬ 
ment  in  motor  imagery  intervention.  The  VMIQ 
was  constructed  specifically  to  test  kinaesthetic 
imagery  ability  (the  ability  to  "visualize"  and  "feel" 
movement)  and  contains  24-item  scale  consist¬ 
ing  of  movements  that  the  subject  is  requested 
to  imagine.  The  questionnaire  incorporates  a 
variety  of  relatively  simple  upper-extremity, 
lower-extremity,  and  whole-body  movements. 
The  best  attainable  score  is  120,  and  the  worst 
obtainable  score  is  24.  VP  scored  120  on  the 
VMIQ,  showing  good  movement  imagery  skills. 
Further  neuropsychological  examination  ex¬ 
cluded  motor  apraxia,  ideomotor  apraxia,  and 
ideational  apraxia. 

Testing  of  motor  abilities 

VP  was  tested  5  times:  (1)  2  weeks  before  the 
first  practice  session  (baseline  assessment);  (2) 
at  the  beginning  of  the  intervention  period;  (3) 
1,5  weeks  after  starting  Ml  practice  (midterm 
evaluation);  (4)  at  the  end  of  the  intervention 
period;  (5)  1  month  after  hospital  practice  termi¬ 
nation.  Further  follow-up  evaluations  are  sched¬ 
uled  12  and  24  weeks  after  practice  termination. 
VP  was  evaluated  for  level  of  impairment  and 
disability.  Pretreatment  and  posttreatment  meas¬ 
ures  were  the  Fugl-Meyer  Assessment  of  Sen¬ 
sorimotor  Impairment  (Fugl-Meyer  Scale),  and 
the  Action  Research  Arm  Test  (ARA).  The  Fugl- 
Meyer  Scale'^  has  been  used  extensively  as  a 
measure  of  impairment  in  studies  measuring 
functional  recovery  in  patients  with  strokes.  Its 
primary  value  is  the  100-point  motor  domain, 
which  has  received  the  most  extensive  evalua¬ 
tion.^®  The  specific  items  in  the  upper-extremity 
subsections  were  derived  from  the  Brunnstrdm 
stages  of  poststroke  motor  recovery.  The  upper- 
extremity  motor  component,  which  consists  of  66 
points,  was  used  in  this  study.  The  Action  Re¬ 
search  Arm  Test^°  is  an  outcome  measure  de¬ 
signed  specifically  for  use  with  patients  with 
strokes.  The  test  is  divided  into  4  categories 
(grasp,  grip,  pinch,  and  gross  movement),  with 
each  item  graded  on  a  4-point  scale  (0=can  per¬ 
form  no  part  of  the  test,  1=performs  test  partially, 


2=completes  test  but  takes  abnormally  long  time 
or  has  great  difficulty,  3=performs  test  normally) 
and  a  total  possible  score  of  60. 

Intervention 

Based  on  VP's  responses  to  the  VMIQ,  we  de¬ 
termined  that  he  was  able  to  use  both  kinesthetic 
and  visual  forms  of  imagery  equally  well.  There¬ 
fore,  internal  as  well  as  external  imagery  scenes 
were  applied  in  his  intervention  protocol.  The 
intervention  focused  on  the  amelioration  of  spe¬ 
cific  impairments  and  on  improving  speed  and 
precision.  All  inpatient  practice  sessions  were 
conducted  by  the  same  medical  professionals. 
The  coordinator  of  the  practice  session  is  a 
physiatrist,  specialized  in  Orthopedics  and  Trau¬ 
matology,  with  over  25  years  of  clinical  experi¬ 
ence. 

Inpatient  phase:  the  Virtual  Reality  Mirror 

The  inpatient  intervention  consists  of  one  daily 
session,  three  days  a  week,  for  four  consecutive 
weeks.  This  time  frame  was  chosen  because 
according  to  available  literature  the  majority  of 
upper  extremity  rehabilitation  programs  last  4  to 
6  weeks  and  because  substantial  improvement 
has  been  demonstrated  after  4  weeks  of  im¬ 
agery  practice.  Each  therapeutic  session  in¬ 
cluded  Vi.  hour  of  traditional  physiotherapy 
(muscle  stretching  and  Bobath  neuro-motor  re¬ 
habilitation)  plus  Vi  hour  of  VR-Mirror  treatment. 
The  treatment  focused  on  the  following  motor 
exercises:  1)  flexion/extension  of  the  wrist;  2) 
prono-supination  of  the  forearm;  3)  flexion/ 
extension  of  the  elbow  with  (assisted)  stabiliza¬ 
tion  of  the  shoulder.  The  apparatus  used  by  the 
patients  consists  of  a  movement  tracking  system 
and  a  custom-designed  visualization  workbench 
that  we  call  the  Virtual  Reality  Mirror.^'  The  VR 
mirror  displays  a  3D  electronic  image  of  the 
movement  performed  by  the  patient’s  healthy 
limb.  This  is  viewed  from  an  first-person  per¬ 
spective,  which  is  supposed  to  facilitate  the  gen¬ 
eration  of  kinesthetic  motor  imagery.^  Each 
practice  session  with  the  VR-mirror  was  com¬ 
posed  of  four  phases. 

1.  During  the  pre-training  phase  the  patient  re¬ 
ceives  instructions,  which  explain  how  the  treat¬ 
ment  works,  encouraging  the  patient  to  relax 
and  reducing  performance  anxiety. 
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2.  In  the  next,  physical  training  phase  the  thera¬ 
pist  shows  the  patient  how  to  perform  the 
movement  with  the  unaffected  arm.  When  the 
patient  then  performs  the  task,  the  system 
tracks  his  arm  and  creates  a  3D  model  of  the 
movement. 

3.  In  the  next,  imagery  training  phase,  the  pa¬ 
tient  is  asked  to  create  a  mental  image  of  the 
impaired  arm  performing  the  movement,  as 
viewed  from  an  internal  perspective.  When  the 
patient  starts  to  imagine  the  movement,  he 
presses  a  button  (using  her  healthy  hand), 
pressing  it  again  when  he  has  finished.  This 
allows  the  therapist  to  measure  the  time  he 
takes  to  imagine  each  movement  (RT).  Basic 
research  has  shown  that  response  times  for 
physical  and  imagined  movements  are  subject 
to  common  laws  and  principles®:  comparing  a 
patient's  RTs,  while  imagining  and  physically 
performing  the  same  movements,  allows  the 
therapist  to  statistically  assess  the  quality  of  the 
patient's  motor  imagery. 

4.  After  the  patient  has  completed  the  mental 
rehearsal  exercise,  he  is  instructed  to  watch  the 
display  on  the  VR  mirror  (virtual  training  phase). 
The  3D  model  created  earlier  is  used  to  gener¬ 
ate  a  mirror  image  of  the  movement  originally 
performed  by  the  healthy  arm.  After  watching 
the  reflected  limb  on  the  screen,  the  patient  is 
invited  to  physically  perform  the  exercise,  mov¬ 
ing  her  arm  in  time  with  the  mirror  image  (see 
fig.  1).  During  execution  of  the  task,  the  system 
tracks  the  movement  of  the  impaired  arm, 
measuring  the  deviation  between  the  move¬ 
ment  the  patient  performs  with  the  impaired 


arm  and  the  “ideal”  movement,  as  performed  by 
the  healthy  arm.  Using  this  measurement, 
which  is  performed  in  real  time,  the  system  pro¬ 
vides  the  patient  with  visual  feedback  describ¬ 
ing  his  performance  on  the  task. 

Home-based  rehabilitation  exercises 

Home-based  rehabilitation  has  the  advantage 
that  stroke  patients  can  practice  skills  and  de¬ 
velop  compensatory  strategies  in  the  environ¬ 
ment  where  they  normally  live.^®  In  this  proto¬ 
col,  the  equipment  provided  is  a  Home  Portable 
Device  (a  portable  display)  with  multimedia  ca¬ 
pabilities  (see  fig,  2).  The  Home  Portable  De¬ 
vice  displays  a  sequence  of  pre-recorded  mov¬ 
ies,  picturing  the  movement  to  be  trained.  After 
viewing  the  movies,  the  patient  is  instructed  to 
take  a  first  person  perspective  and  to  imagine 
performing  the  task  with  the  impaired  arm.  The 
patient  performs  this  sequence  three  times  a 
week. 

RESULTS 

The  clinical  assessment  scores  are  listed  in  Ta¬ 
ble  1 .  Fugl-Meyer  scores  notable  increased  dur¬ 
ing  the  4  weeks  of  intervention  with  modest  in¬ 
creases  during  the  1  month  follow-up.  Action 
Research  Arm  Test  scores  consistently  in¬ 
creased  during  the  4  weeks  of  intervention. 
There  were  3-point  increases  during  the  1  month 
follow-up.  Moreover,  the  patient  had  appreciable 
increases  in  grip  strength  for  the  affected  right 
limb.  Measurements  of  wrist  function  revealed 
increases  in  range  of  motion  during  the  first 
phase  of  intervention,  with  no  losses  in  move- 


Figure  1.  Pictures  of  the  Virtual  Reality  Mirror  in  action 
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Figure  2.  The  portable  display  used  for  the 
home  rehabilitation  mental  imagery  exercises 


ment  range  occurring  after  the  intervention  was 
completed.  VP  was  highly  motivated  throughout 
the  intervention  period  and  was  present  at  all 
sessions.  From  the  third  week  on,  he  reported 
on  increase  in  self-confidence  using  the  affected 
limb. 

DISCUSSION 

The  purpose  of  this  case  report  was  to  describe 
the  implementation  of  technology-supported 
mental  practice  for  improving  motor  performance 
in  one  individual  with  hemiplegia  of  the  upper 
extremity  following  stroke.  We  found  that  mental 
training  combined  with  physical  training  im¬ 
proved  key  measures  of  sensorimotor  impair¬ 
ments  and  functional  ability.  Furthermore,  these 
results  were  maintained  (and  even  improved)  at 
1  month  after  home  rehabilitation  training,  sug¬ 
gesting  that  home  practice  was  useful.  A  ration¬ 
ale  regarding  the  reason  why  mental  practice 
with  technology  combined  with  physical  practice 
seems  to  be  effective  can  be  found  in  the  motor 
behaviour  and  neuropsychology  literature.  Jack¬ 


son  et  al.'*  have  recently  created  a  model,  com¬ 
paring  the  potential  therapeutic  effects  of  mental 
practice  with  other  forms  of  training.  The  model 
assumes  that  practice  involves  declarative 
knowledge,  non-conscious  processes  and  physi¬ 
cal  execution.  In  their  model,  declarative  knowl¬ 
edge  is  the  explicit  knowledge  that  subjects  need 
before  practicing  a  motor  task  (i.e.,  a  knowledge 
of  the  sequence  of  movements  to  be  performed); 
skills  that  are  not  directly  accessible  to  verbal 
description,  such  as  the  timing  of  motor  re¬ 
sponses  to  cues  or  the  co-articulation  of  small 
segments  of  movement  and  the  rapid,  sequential 
activation  or  inhibition  of  different  muscle  groups 
are  regulated  by  non-conscious  processes: 
physical  execution  is  the  musculoskeletal  activity 
necessary  to  carry  out  the  intended  action.  Ac¬ 
cording  to  Jackson  and  his  collaborators,  these 
different  levels  of  processing  interact  in  different 
ways  in  different  forms  of  training.  Physical  prac¬ 
tice  involves  all  three  levels  of  processing;  in 
mental  practice,  on  the  other  hand,  learning  de¬ 
pends  crucially  on  the  interaction  between  de¬ 
clarative  knowledge  and  non-conscious  proc¬ 
esses;  in  physical  practice  it  is  possible  to  learn 
a  motor  task  implicitly;  in  mental  practice  sub¬ 
jects  have  a  good  declarative  knowledge  of  the 
different  components  of  the  task  before  they 
start  practicing.  The  similarity  between  the  cir¬ 
cuitry  involved  in  imagining  and  executing  move¬ 
ments  suggests,  however,  that  the  neuronal  net¬ 
work  implicated  in  the  non-conscious  aspects  of 
a  task  can  be  primed  as  effectively  through  men¬ 
tal  as  through  physical  practice  (p.  1138).  In  ad¬ 
dition,  the  model  predicts  that  internally  driven 
images,  promoting  kinesthetic  “sensations”  of 
movement,  could  be  highly  effective  in  activating 
the  non-conscious  processes  involved  in  motor 
training. 


Test 

Baseline 

Inpatient  phase 

Home  reha¬ 
bilitation 
phase 

1  week 

Midterm 

4  weeks 

1  month 

20/66 

20/66 

27/66 

34/66 

36/66 

Action  Research  Arm 
Test 

MI60 

12/60 

20/60 

26/60 

29/60 

Table  1.  Functional  Improvements  After  Training 
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According  to  Jackson  and  his  colleagues,  this 
would  explain  why  kinesthetic  imagery  is  more 
effective  than  purely  visual  imagery.  Their  model 
proposes  to  conceptualize  mental  practice  with 
motor  imagery  as  a  means  to  access  the  other¬ 
wise  non-conscious  learning  processes  involved 
in  a  task.  They  recognize,  however,  that  the  ab¬ 
sence  of  direct  feedback  from  physical  execution 
makes  mental  practice  on  its  own  a  less  effective 
training  method  than  physical  practice. 

In  conclusion,  results  of  this  pilot  study  indicate 
that  the  stimulation  of  mental  practice  through 
the  use  of  advanced  visualization  technology, 
combined  with  physical  practice,  leaded  to  func¬ 
tional  gains  in  the  paretic  upper  extremity  of  a 
patient  with  stable  motor  deficit  of  the  upper  right 
limb  following  stroke.  Although  a  single-case 
study  does  not  allow  us  to  determine  precisely 
the  contribute  of  mental  practice  and  which  pa¬ 
rameters  of  the  training  are  most  useful,  a  future 
goal  is  to  systematically  define  the  most  effica¬ 
cious  protocol  for  each  patient.  Finally,  random¬ 
ized  clinical  studies  are  needed  to  assess 
whether  this  experimental  protocol  durably  im¬ 
proves  upper  extremity  motor  function  in  hemi¬ 
plegic  patients  and  how  these  functional  motor 
adaptations  are  mediated  by  central  neural 
mechanisms. 
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Use,  Understanding  and  Learning  in  Virtual  Environments  By  Adoles¬ 
cents  With  Autistic  Spectrum  Disorders 

Sarah  Parsons 

School  of  Education,  University  of  Birmingham 

Abstract;  There  has  been  increasing  interest  over  recent  years  in  the  application  of  virtual  reality 
technology  to  facilitating  skills  and  understanding  for  children  with  autistic  spectrum  disorders 
(ASDs).  This  paper  presents  a  summary  of  a  series  of  studies  that  sought  to  investigate  three  main 
research  questions;  would  adolescents  with  ASDs  be  able  to:  (1)  Use  Virtual  environments  (VEs) 
appropriately?  (2)  Understand  them  as  representational  devices?  (3)  Learn  new  information  from 
VEs  about  social  skills?  Overall  findings  suggest  that  some  adolescents  with  ASDs  can  use,  under¬ 
stand  and  learn  social  skills/conventions  presented  in  VEs  although  some  students  with  lower  verbal 
IQ  and  weak  executive  skills  benefited  less  from  this  kind  of  intervention.  An  important  consideration 
for  future  research  is  the  role  of  the  facilitator  in  ‘scaffolding’  participants’  use  of  the  VEs.  If  the  aim  is 
to  provide  effective  educational  support,  to  what  extent  is  it  useful  to  try  to  separate  the  effect  of  the 
VE  from  the  role  of  a  facilitator? 


INTRODUCTION 

At  the  time  of  writing,  even  a  cursory  search  on 
the  Internet  for  “Autism  and  Virtual  Reality”  re¬ 
veals  a  wealth  of  hits  reflecting  the  growing  in¬ 
terest  in  the  application  of  advanced  technology 
to  the  educational  realm  of  children  with  autistic 
spectrum  disorders  (ASDs).  According  to  the 
National  Autistic  Society  (NAS)  in  the  UK  this  is 
part  of  a  trend  that  has  recognised  the  benefits 
of  computers  more  generally  for  people  diag¬ 
nosed  with  the  disorder,  which  is  characterised 
by  severe  social  and  communicative  deficits: 

“Because  computers  offer  a  context-free 
environment  in  which  many  people  with 
autism  feel  comfortable,  therapists  and 
teachers  are  increasingly  using  virtual  real¬ 
ity  tools  to  teach  life  skills,  such  as  crossing 
the  road,  and  social  skills,  such  as  recog¬ 
nising  emotions  in  other  people”^® 

However,  despite  the  burgeoning  number  of 
Internet  hits,  there  has  actually  been  little  in  the 
way  of  published,  peer-reviewed,  systematic 
research  investigating  the  usability  and  poten¬ 
tial  of  virtual  environments  (VEs)  specifically  for 
people  with  ASDs.  There  are  good  reasons  for 
wanting  to  know  more  about  this  topic  though 
because  there  are  ostensible  pros  and  cons  to 
the  approach.  On  the  plus  side,  there  is  growing 
evidence  that  computer-based  educational  ap¬ 
plications  have  resulted  in  significant  improve¬ 


ments  in  the  target  behaviours  or  learning  ob¬ 
jectives  for  children  with  autism'’’^®'^^'^^'^  which 
suggests  that  virtual  reality  could  be  a  useful 
addition  to  any  educational  ICT  toolkit.  Doubts 
about  the  usefulness  of  VR  for  people  with 
ASDs  arise  though  when  considering  a  particu¬ 
lar  aspect  of  the  disorder:  the  difficulty  In  under¬ 
standing  people’s  behaviour  and  intentions  in 
the  real  world,  let  alone  within  the  realms  of  a 
virtual  one^®.  It  perhaps  seems  counterintuitive 
to  facilitate  social  understanding  or  communica¬ 
tion  in  the  real  world  by  immersing  people  in  a 
virtual  one! 

However,  the  optimism  for  using  VEs  with  peo¬ 
ple  with  ASDs  is  based  on  an  important  prem¬ 
ise  which  is  that  interactions  or  experiences  in  a 
virtual  world  may  be  less  threatening  for  people 
with  ASDs  compared  to  the  real  world  because 
many  of  the  inputs  of  real  world  interaction 
(which  people  with  ASDs  often  find  very  confus¬ 
ing)  can  be  directly  controlled  or  manipulated. 
This  potentially  allows  a  forum  where  behav¬ 
iours  and  responses  can  be  practised  and  built- 
upon  in  a  context  that  shares  some  similarities 
with  the  real  world,  thereby  offering  greater  po¬ 
tential  for  generalisation.  As  the  NAS  note 
though,  the  most  important  feature  of  VR  is 
safety:  “Above  all,  VR  offers  a  safe  learning 
environment  in  which  the  individual  may  make 
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Choose  level  of  diffi 


taneously 


Click  to  sit  down  at 
the  table 


Icons  appear  here  to 
pay  for  items 


Click  to  ask  a  ques¬ 
tion  (or,  when  a 
thought  bubble  ap¬ 
pears,  find  out  what 
another  person  may 
be  thinking) 


Sal  at  the  table  is  someone  voii  don't  know 


What  shonld  you  do? 


DIKrtCtJLTY 


Click  to  hear  overall 
task 


culty  within  the  VE 
by  pressing  corre¬ 
sponding  number  on 
keyboard 


Tasks  and  speech  are 
heard  and  read  simul- 


Figure  1.  Screen  shot  of  cafe  VE  illustrating  the  key  functions  of  the  program 


mistakes  which  might  be  physically  or  socially  (for  background  information  on  AS  Interactive 
hazardous  in  the  real  world”  see  Beardon  et  al.,  2001;  Parsons  et  al.,  2000) 

we  made  a  start  at  investigating  some  of  these 
The  pioneering  work  of  Strickland^^' provided  issues  and  aimed  to  answer  three  main  re- 

an  initial  demonstration  that  young  children  with  search  questions:  would  adolescents  with  ASDs 

autism  could  respond  appropriately  to  a  virtual  be  able  to  (1)  use  the  VEs  appropriately?  (2) 

scene  presented  in  a  head-mounted  display.  understand  the  VEs  as  representational  de- 

However,  the  small  sample  size,  coupled  with  vices?  (3)  learn  new  information  from  VEs 

concerns  over  the  acceptability  of  the  heavy  about  social  skills? 

headset  and  lack  of  any  comparison  partici¬ 
pants  meant  that  the  results  were  limited.  Other  The  main  studies  and  findings  are  summarised 

published  articles  considered  the  value  of  VEs  below,  followed  by  some  speculation  about  the 

for  people  with  ASDs®  but  (at  the  time)  con-  direction  of  future  research  in  this  field, 

tained  no  practical  demonstrations  of  their  use. 

Consequently,  there  was  a  clear  need  to  con-  SUMMARY  OF  GENERAL  METHODOLOGY 
duct  some  systematic  research  to  consider  fun¬ 
damental  issues  about  usability  and  interpreta-  In  total,  there  were  two  types  of  VEs  -  a  cafe 

tion;  in  particular,  whether  people  with  ASDs  and  a  bus  -  both  of  which  were  presented  to 

could  understand  a  virtual  environment  as  participants  on  a  laptop  (with  a  Pentium  3 

something  that  represented  reality;  that  is,  un-  650MHz  Processor,  64  Mb  RAM  and  Windows 

derstand  important  differences  between  virtual  98),  but  could  also  be  used  on  standard  desk- 

and  real  worlds  but  also  realise  that  virtual  envi-  top  PCs.  VEs  were  navigated  with  a  USB  joy- 

ronments  have  enough  similarities  to  the  real  stick,  and  objects  selected  with  a  standard 

world  to  usefully  inform  them  about  the  latter.  mouse.  Not  all  studies  used  both  VEs;  initially 
In  a  three-year  multidisciplinary  research  pro-  the  cafe  environment  was  used  and  the  bus 

ject  based  at  the  University  of  Nottingham,  UK  environment  added  in  later  studies.  Environ- 
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merits  were  built  using  Superscape  Version  5.5 
and  run  using  the  Superscape  Visualiser.  The 
VEs  were  ‘single-user’,  meaning  that  only  one 
person  could  navigate  the  scene  at  any  one 
time.  The  content  of  the  cafe  and  bus  environ¬ 
ments  were  developed  from  a  series  of  design 
iterations'^’ so  we  were  confident  that  the 
scenarios  were  easily  recognisable  and  usable 
for  our  target  participants.  Feedback  and  in¬ 
structions  for  users  were  provided  through  tex¬ 
tual  and  audible  prompts  from  the  programme. 
There  were  also  a  number  of  functions,  repre¬ 
sented  by  icons  on  the  screen,  which  users 
could  access,  such  as  clicking  on  a  speech- 
bubble  icon  to  ask  a  question  (see  Figure  1). 

SUMMARY  OF  STUDIES  AND  MAIN  FIND¬ 
INGS 

Study  1 :  The  first  formal  study  included  a  group 
of  adolescents  with  ASDs,  and  two  non-autistic 
comparison  groups  -  one  matched  according  to 
verbal  IQ  (mean  =  69)  and  the  other  to  non¬ 
verbal  IQ  (mean  =  92).  Participants  were  seen 
individually  and  used  a  training  VE,  which  was 
a  single-user  environment  navigated  with  a  joy¬ 
stick  and  interacted  with  via  a  standard  mouse. 
The  training  environment  consisted  of  a  simple 


building  surrounded  by  open  space  (see  Figure 
2a).  Inside  the  building,  there  were  maze-like 
corridors  (see  Figure  2b).  Moving  around  the 
outside  and  inside  of  the  building  enabled  par¬ 
ticipants  to  practice  manoeuvring  in  open  and 
confined  spaces,  respectively.  A  large  room 
was  at  the  end  of  the  corridor,  which  contained 
eight  different  objects.  The  objects  could  be 
seen  only  as  a  green  square  from  a  distance, 
but  were  revealed  automatically  as  the  user 
moved  closer  (Figure  2c).  Clicking  on  an  object 
with  the  mouse  moved  the  item  to  the  black 
display  bar  at  the  bottom  of  the  screen  (see 
Figure  2c  in  which  the  kettie  has  already  been 
selected).  There  were  five  training  environ¬ 
ments  in  total  and  participants  were  required  to 
find  a  series  of  four  objects  in  a  randomised 
order,  shown  on  a  card  visible  throughout  the 
procedure  (for  further  details  see  Parsons, 
Mitchell  &  Leonard,  2004). 

Over  four  separate  trials,  participants  with 
ASDs  improved  significantly  in  the  time  spent 
completing  these  tasks,  compared  to  the  VIQ- 
match  group,  who  did  not  improve  significantly. 
Crucially,  by  the  end  of  the  four  trials,  the  par¬ 
ticipants  with  ASDs  were  not  significantly  differ- 


2  (a)  Outside  view  of  the  training  environment 
2  (b)  Corridor  in  the  training  environment 

2  (c)  Near  and  distant  views  of  objects  in  the  training  environment 
2  (d)  Scene  from  the  Virtual  Cafe 
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A. 


C. 


Figure  3(a)  starting  point;  3(b)  approaching  the  cafe;  3(c)  approaching  the  counter 


ent  from  the  PIQ-match  group  in  time  taken  to 
complete  the  tasks. 

In  addition,  all  participants  were  required  to 
complete  a  number  of  tasks  in  a  virtual  cafe  (an 
earlier  version  to  that  depicted  in  Figure  1  taken 
from  Brown  et  al.,  1999),  such  as  find  some¬ 
where  to  sit,  order  and  pay  for  some  food,  and 
move  to  the  counter  to  order  a  drink.  Partici¬ 
pants  were  also  asked  to  make  links,  as  well  as 
suggest  differences,  between  virtual  scenes 
encountered  in  the  virtual  cafe  as  well  as  a 
similar  videoed  scene  including  real  people  in  a 
real  bar.  A  video  representation  is  different  from 
virtual  reality  in  the  sense  that  it  is  a  photo¬ 
realistic  representation,  which  is  familiar  to  peo¬ 
ple  through  the  use  of  television  and  other  vid¬ 
eos.  A  virtual  environment,  by  contrast,  is  less 
familiar  to  many  people  and  is  a  more  abstract 
form  of  representation.  Comparing  these  repre¬ 
sentations  enabled  us  to  see  whether  partici¬ 
pants  interpreted  them  in  similar  ways.  The  ma¬ 
jority  of  participants  with  ASDs  understood  the 
VE  as  something  that  represented,  but  was  not 
identical  to,  reality.  That  is,  they  said  that  virtual 
people  in  the  scene  were  talking  to  each  other 
or  to  the  barman  even  though  they  were  not 
actually  talking,  but  were  oriented  towards  each 
other  in  such  a  way  that  suggested  interper¬ 
sonal  interaction.  In  addition,  the  majority  of 
participants  with  ASDs  also  acknowledged  im¬ 
portant  differences  between  the  VE  and  the 
video;  that  the  people  in  the  VE  were  not  real 
but  the  people  in  the  video  were. 

Most  participants  also  completed  the  tasks  in 
the  virtual  cafe  successfully,  but  the  groups  dif¬ 
fered  significantly  in  one  important  respect. 
When  approaching  the  counter  to  buy  a  drink, 
the  participants  with  ASDs  were  judged  (by  na¬ 
ive  observers)  as  significantly  more  likely  to 
move  very  close  to,  or  walk  between  a  virtual 


couple  stood  at  the  bar,  compared  to  both  the 
VIQ-  and  PlQ-matched  groups.  This  behavior 
seems  socially  inappropriate,  especially  as 
there  was  a  large  empty  space  at  the  bar,  to  the 
right  of  the  couple  (see  Figure  2d).After  ruling 
out  a  number  of  explanations,  such  as  weak 
navigational  and  executive  skills  (behaviours 
involved  in  sequencing,  planning  and  impulse 
control),  it  seemed  possible  that  the  participants 
with  ASDs  might  struggle  more  than  others  with 
understanding  the  importance  of  personal 
space  in  social  situations  (at  least  as  depicted 
in  a  virtual  environment),  despite  understanding 
the  appropriate  representational  character  of 
VEs. 

Study  2:  A  second  stud/"'  explored  whether 
participants  with  ASDs  adhered  to  social  con¬ 
ventions  generally  in  VEs,  such  as  avoiding 
walking  across  a  neighbour’s  garden  and  re¬ 
specting  the  personal  space  of  people  ostensi¬ 
bly  engaged  in  conversation  (see  Figure  3). 

The  majority  of  participants  with  ASDs  behaved 
in  a  similar  way  to  non-autistic  comparison 
groups  by  treating  the  VE  like  a  game  in  most 
situations.  That  is,  they  often  walked  between 
the  two  virtual  people  en  route  to  a  counter  in  a 
virtual  cafe,  however  members  of  the  ASD 
group  were  less  likely  to  remark  that  this  was 
socially  unacceptable  than  students  in  the  com¬ 
parison  groups.  Students  with  ASDs  were  also 
more  likely  to  walk  across  a  neighbour’s  garden 
compared  to  the  PIQ-match  group,  and  a  third 
of  the  ASD  group  (4  out  of  12)  showed  substan¬ 
tial  ‘off-task’  behaviour.  This  involved  them 
walking  around  the  caf§,  sometimes  even  be¬ 
hind  the  bar,  and  navigating  up  to  other  people 
in  the  scene.  This  behaviour  was  shown  by  stu¬ 
dents  with  low  VIQ  and  weak  executive  abilities 
(as  assessed  by  the  Behavioural  Assessment 
of  the  Dysexecutive  Syndrome;  BADS^®),  and 
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only  by  a  subset  within  the  ASD  group.  People 
in  this  subset  were  also  unable/unwilling  to 
comment  on  the  links  between  the  VE  and  ‘real 
world’  behaviour,  even  when  compared  to  the 
VIQ-match  group.  Thus,  it  is  possible  that  a  mi¬ 
nority  of  students  with  ASDs  may  struggle  es¬ 
pecially  with  understanding  how  the  VE  can 
inform  them  about  reality. 

By  contrast,  many  of  the  comparison  partici¬ 
pants,  but  none  in  the  ASD  group,  reflected  on 
the  fact  that  they  would  behave  differently  in 
real  life  because  it  was  not  socially  appropriate 
to  walk  through  people’s  gardens,  or  between 
people  having  a  conversation,  for  example: 

‘In  real  life,  I  wouldn’t  go  though  the  middle, 
but  because  it’s  VR  you  treat  it  differently  - 
you  know  it’s  not  real.  [I  was]  just  interested 
in  the  completion  of  the  task’  -  PIQ-match 
student. 

Interestingly,  although  In  the  minority,  some 
participants  who  avoided  walking  between  the 
two  people  en  route  to  the  counter  were  very 
sensitive  to  the  social  relevance  of  the  virtual 
cafe  (including  some  of  the  ASD  students),  for 
example: 

’...  if  there  were  two  people  facing  each 
other,  it  looked  like  they  were  having  a  con¬ 
versation.  It  would  not  be  very  good  to  walk 
through.  If  they  had  their  backs  to  each 
other  it  would  not  make  much  difference 
which  way  I  was  going’  -  ASD  student. 

Thus,  some  of  the  participants  with  ASDs 
showed  a  good  understanding  of  particular  so¬ 
cial  conventions  in  the  VE  -  namely,  not  walk¬ 
ing  across  neighbours’  gardens  and  not  walking 
between  people  having  a  conversation.  Never¬ 
theless,  these  individuals  were  in  the  minority 
within  the  group  -  with  the  remainder  unwilling 
to  respond  to  questions  or  seemingly  unaware 
of  the  real  world  implications  of  their  behaviour. 
This  was  in  contrast  to  the  VIQ-  and  PIQ- 
matched  groups,  who  seemed  more  willing  to 
use  the  path/pavement  at  the  start  of  the  task 
and  more  aware  of  how  their  behaviour  in  the 
VE  related  to  real  world  social  conventions. 
Taken  together,  the  first  two  studies  provided 
some  in-depth  information  about  how  a  group  of 
adolescents  with  ASDs  use  and  interpret  VEs. 


In  particular,  there  was  evidence  to  suggest  that 
they  were  on  a  par  with  typically-developing 
counterparts  in  their  ability  to  use  the  VEs,  and 
understood  the  scenes  as  appropriately  repre¬ 
sentational.  By  contrast,  the  main  weaknesses 
for  participants  with  ASDs  seemed  to  be  in  the 
social  domain  -  reinforcing  the  view  that  VEs 
could  offer  some  support  in  this  area  by  pre¬ 
senting  scenes  and  ideas  that  were  not  already 
well  within  the  grasp  of  most  students  with 
ASDs.  Thus,  a  remaining  crucial  question  was 
whether  participants  with  ASDs  could  learn  spe¬ 
cific  social  conventions  using  the  technology. 

Study  3:  This  was  addressed  in  a  third  study’^, 
which  included  a  group  of  six  adolescents  with 
ASDs.  Participants  used  a  new  version  of  the 
virtual  cafe  with  the  learning  objectives  of  find¬ 
ing  an  appropriate  place  to  sit,  and  asking  ap¬ 
propriate  questions  (Figure  1).  Specifically,  the 
aim  was  to  help  students  understand  that  if 
there  is  a  completely  empty  table  available  in  a 
cafe  then  it  is  more  appropriate  to  sit  at  it  than 
with  strangers,  but  if  there  are  no  completely 
empty  tables  free  then  it  is  appropriate  to  ask  a 
question  before  sitting  down  (e.g.’Is  it  OK  if  I  sit 
here?).  The  virtual  cafe  incorporated  four  differ¬ 
ent  levels  of  difficulty  as  the  user  progressed; 
the  cafe  became  increasingly  busy  and  noisy, 
with  fewer  choices  of  appropriate  seating.  Par¬ 
ticipants  used  the  VE,  with  the  support  of  a  fa¬ 
cilitator  (the  researcher  in  this  case)  over  a 
number  of  sessions.  They  were  also  asked  to 
make  judgments  about  where  they  would 
choose  to  sit,  and  why,  in  video  clips  of  real 
cafes.  The  video  clips,  whilst  not  ideal,  served 
as  approximations  to  judgements  about  the  real 
world.  These  were  presented  to  participants  for 
comment  before  and  after  VE  use  and  naive 
coders  were  asked  to  rate  the  explanations  for 
social  appropriateness. 

Results  showed  that  four  of  the  six  participants 
improved  in  their  awareness  of  social  conven¬ 
tions,  both  in  terms  of  performance  in  the  virtual 
cafe  and  in  judgments  and  comments  made  in 
relation  to  the  video  clips.  That  is.  a  majority  of 
participants  were  judged  as  providing  more  so¬ 
cially  appropriate  and  insightful  comments 
about  where  they  would  sit  and  why  in  relation 
to  the  video  scenes,  specifically  following  a  ses¬ 
sion  with  the  VE.  An  example  of  high-scoring 
reasoning  was:  “because  no-one’s  sitting  there. 
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I  won’t  be  in  anyone’s  way”,  and  of  low-scoring 
reasoning:  “so  1  can  look  at  the  windows”. 

The  design  of  the  study  allowed  us  to  pinpoint 
the  improvement  in  understanding  to  the  spe¬ 
cific  time  of  VE  use,  and  not  to  general  im¬ 
provement  in  understanding  over  time.  This 
was  because  whilst  all  participants  received  the 
same  tasks,  they  were  completed  in  a  different 
order;  for  example,  half  the  participants  had 
sessions  with  the  VE  between  the  video  meas¬ 
ures  at  Time  1  and  Time  2,  whilst  the  remainder 
had  their  VE  sessions  between  the  video  meas¬ 
ures  at  Time  2  and  Time  3.  Gains  in  under¬ 
standing  should  be  apparent  directly  following 
VE  but  not  necessarily  during  testing  sessions 
that  did  not  directly  follow  the  VE.  Importantly, 
the  facilitator  played  a  crucial  role  in  the  suc¬ 
cess  of  this  approach,  by  helping  the  user  to 
interpret,  understand  and  discuss  the  social 
scenes  depicted  in  the  VE. 

Study  4:  Finally,  after  demonstrating  successful 
outcomes  in  terms  of  use,  understanding  and 
learning  in  the  first  three  studies,  it  was  impor¬ 
tant  to  highlight  the  views  of  people  with  ASDs 
directly  to  gain  a  sense  of  how  they  viewed  the 
VEs  and  whether  they  thought  VEs  could  be 
useful  to  them  in  any  way.  Therefore,  in  the  fi¬ 
nal  study  we  adopted  a  case  study  approach 
with  two  adolescent  males  with  an  ASD  and 
sought  their  opinions  and  comments  over  a 
number  of  sessions  with  the  VEs,  and  during  an 
informal  interview^®.  Both  students  learned  the 
target  social  conventions  (these  were  the  same 
as  the  third  study)  over  the  period  of  VE  use, 
with  the  aid  of  rich  discussions  with  the  facilita¬ 
tor  about  social  aspects  of  behaviour  and  what 
other  people  (in  the  virtual  scene)  could  be 
thinking.  In  addition,  both  students  commented 
on  how  the  VE  could  be  relevant  to  their  lives, 
and  one  student  excitedly  remarked  how  he 
had  used  the  information  he  had  learned  from 
the  VE  in  a  real  life  situation.  He  was  travelling 
on  a  busy  train  in  the  summer  break  and  there 
were  few  empty  seats,  so  he  asked  a  man  if  the 
seat  beside  him  was  free.  The  student  said  that 
he  felt  proud  about  asking  the  question  be¬ 
cause  this  was  not  something  he  would  have 
usually  done.  In  addition,  he  said  how  much  the 
VE  had  helped  him  “Cause  it  could  help  me 
learn  what  polite  and  sensible  things  to  do  in 
public  places”  (student  with  ASD,  aged  14). 


CONCLUSIONS 

Whilst  anecdotal,  comments  like  the  one  in¬ 
cluded  above  are  extremely  encouraging  and 
add  to  our  growing  body  of  knowledge  about 
whether,  and  how,  people  with  ASDs  can  use, 
understand  and  learn  from  VEs  depicting  social 
situations.  Overwhelmingly,  the  emerging  pic¬ 
ture  from  the  four  studies  summarised  here  is  a 
positive  one,  which  can  continue  to  be  built 
upon  in  any  future  work.  This  is  especially  en¬ 
couraging  in  the  context  of  fairly  limited  func¬ 
tionality  in  the  VE  and  the  rather  unconvincing 
representations  of  people  in  the  scene,  which 
were  blocky  and  cartoon-like  in  appearance. 
Despite  such  limitations,  many  of  the  partici¬ 
pants  assigned  representational  qualities  to  the 
scene  (being  like  reality  in  some  respects  but 
not  identical  to  it)  and  enjoyed  using  the  VEs. 
An  important  point  to  note  too  is  that  the  VEs 
and  learning  objectives  were  designed  with  the 
input  of  teachers  working  with  the  participants 
and  so  were  based  on  clearly  identified  needs. 
This  was  an  essential  part  of  the  project  and 
ensured  that  the  research  put  the  needs  of  par¬ 
ticipants  first  rather  than  pursuing  worthy  but 
perhaps  esoteric  agendas  of  the  research  team! 
Virtual  environments  and  programming  exper¬ 
tise  can  now  accommodate  far  more  sophisti¬ 
cated  levels  of  representation  as  well  as  more 
finely  tuned  aspects  of  interpersonal  behaviour 
including  direction  of  gaze  and  conversation. 
Recent  work  from  David  Moore  and  his  stu¬ 
dents  in  the  UK,  for  example,  has  utilised  col¬ 
laborative  virtual  environments  (CVEs)  for  in¬ 
vestigating  the  usefulness  of  emotionally  ex¬ 
pressive  avatars  in  communicative  contexts  for 
children  with  autism®  as  well  as  considering 
aspects  of  gender,  personal  space  and  anxiety, 
which  could  eventually  be  useful  for  children 
with  autism^^.  The  use  of  CVEs  allows  great 
potential  for  developing  social  and  communica¬ 
tive  skills  in  more  dynamic  (yet  safe)  environ¬ 
ments  than  has  hitherto  been  possible,  thus 
offering  greater  potential  for  learning  skills  that 
are  likely  to  be  of  use  in  the  real  world. 

It  is  an  exciting  time  for  researchers  because 
the  potential  for  creating  really  beneficial  out¬ 
comes  for  children  with  ASDs  is  a  distinct  possi¬ 
bility,  although  of  course  there  is  still  a  great 
deal  to  be  done  in  research  terms.  There  is  also 
a  need  for  researchers  to  be  clear  about  their 
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aims  and  why  participants  with  clinical  diagno¬ 
ses  (including  autism)  are  involved  in  studies;  it 
could  be  too  easy  to  pursue  a  line  of  investiga¬ 
tion  because  it  seems  timely  and  interesting  but 
ultimately  of  limited  benefit  to  participants. 
There  are  essentially  three  main  potential  foci 
for  ‘Autism  and  Virtual  Reality’  research:  what 
participants  with  autism  can  teli  us  about  VR; 
what  VR  can  tell  us  about  autism  and  what  VR 
can  offer  people  with  autism  in  terms  of  educa¬ 
tional/learning  needs. 

In  terms  of  what  participants  with  autism  can  tell 
us  about  VR,  there  are  substantial  ethical  con¬ 
cerns  about  including  people  with  clinical  disor¬ 
ders  in  research  where  the  technology  rather 
than  the  people  is  the  primary  focus.  For  exam¬ 
ple,  the  unique  perspective  that  people  with 
ASDs  may  have  on  the  (real)  world  could  tell  us 
something  interesting  about  the  nature  of  VEs 
themselves,  but  is  it  ethically  justifiable  to  pur¬ 
sue  such  a  question  by  itself?  Similarly,  there  is 
a  query  about  what  VR  stands  to  tell  us  about 
the  condition  of  autism  per  se?  Despite  sub¬ 
stantial  research  on  how  non-autistic  members 
of  the  population  use,  interpret  and  understand 
VEs,  the  links  between  virtual  and  real  world 
behaviours  are  still  not  properly  understood, 
including  when  and  why  people  choose  to  apply 
real  world  conventions  to  virtual  contexts  and 
when  they  do  not.  Do  we  know  enough  about 
the  use  of  VEs  more  generally  to  draw  many 
conclusions  about  how  or  why  participants  with 
ASDs  might  respond  in  virtual  contexts  and  how 
that  links  to  real  world  understanding?  Of 
course,  answers  to  these  questions  may  arise 
in  the  context  of  a  project  with  a  more  educa¬ 
tional  focus  (and  this  was  certainly  the  case  for 
the  AS  Interactive  project)  and  would  be  inter¬ 
esting  to  know,  but  I  would  question  using  the 
above  two  foci  as  starting  points.  Instead,  they 
could  be  questions  that  are  tackled  en  route  to 
a  more  outcome  focused  research  approach, 
which  provides  something  (however  imperfect 
and  iimited)  back  to  the  relevant  participant 
groups. 

The  main  point  is  that  the  potential  of  VEs  for 
providing  powerfui  learning  contexts  for  people 
who  would  really  benefit  from  them  should  be 
the  primary  aim  and  driving  force  of  research  in 
this  area.  The  question  of  ‘what’s  in  it  for  the 
participants?’  should  be  at  the  forefront  of  re¬ 


searchers’  minds  as  they  contemplate  how  to 
take  this  work  forward.  This  may  mean  adopting 
research  approaches  that  take  into  account  the 
importance  of  context  and  interaction  around 
the  computer  when  considering  whether  and 
how  learning  takes  place.  The  challenge  of  this 
became  most  apparent  for  me  in  discussions 
about  the  role  of  a  facilitator  in  the  use  of  the 
VEs;  that  is,  someone  who  sat  alongside  the 
participant  and  guided  them  verbally  through 
the  social  situations  depicted,  perhaps  asking 
the  participant  to  clarify  why  they  took  that  route 
or  talking  through  the  motive  or  reasoning  for 
other  people  behaving  in  a  particular  way  (see 
Parsons  et  al.,  in  press  and  Mitchell  et  al.,  sub¬ 
mitted). 

Quite  clearly,  from  an  experimental  research 
point  of  view,  it  becomes  impossible  to  say 
what  was  really  helping  to  do  the  job  of  learn¬ 
ing;  was  it  the  ‘experiential’  nature  of  the  VE 
and  the  opportunity  to  practice  the  behaviour 
repeatedly,  or  the  interpretation  and 
‘scaffolding’  offered  by  the  facilitator,  or  a  com¬ 
bination  of  both?  I  would  argue  that,  for  me  at 
least,  it  doesn’t  really  matter  all  that  much  be¬ 
cause  learning  coincided  with  use  of  the  VE 
and  so  there  was  something  about  the  context 
that  was  useful  and  important  (Mitchell  et  al., 
submitted).  Moreover,  as  Crook  (1991)  argues, 
the  context  of  learning  (e.g.  within  a  classroom) 
should  be  a  central  consideration  to  the  design 
and  use  of  ICT  resources  because  it  is  always 
within  such  a  context  that  we  would  expect  the 
real  world  outputs  of  our  research  to  be  used  to 
(hopefully)  beneficial  effect: 

“In  order  to  expose  where  and  how  computers 
are  proving  potent,  the  design  of  evaluations 
must  be  sensitive  to  events  that  are  taking 
place  in  the  ‘periphery’  of  the  individual  learner- 
computer  interaction.  In  particular,  evaluation 
should  be  amenable  to  the  idea  that  properties 
of  any  computer  application  include  the  charac¬ 
ter  of  social  processes  that  it  supports  during 
the  circumstances  of  use”  ®. 

Separating  out,  or  attempting  to  narrow  down, 
the  specific  factors  that  promote  learning  may 
only  be  relevant  if  we  are  interested  in  claiming 
that  VEs  are  better  than  other  formats/tools/ 
contexts  for  facilitating  learning  and  I  would 
question  the  usefulness  of  such  a  claim  or  pur- 
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suit  of  one.  For  if  we  are  in  the  business  of  put¬ 
ting  the  needs  of  participants  first  and  pursuing 
a  research  agenda  that  could  be  of  substantial 
educational  benefit  to  people  with  ASDs,  then  I 
think  that  VEs  as  an  option  amongst  many  for 
the  promotion  of  learning  is  not  only  good 
enough  but  also  appropriate  and  desirable.  This 
will  help  to  ensure  that  teachers,  parents  and 
therapists  can  be  equipped  with  a  toolkit  of  pos¬ 
sibilities  that  could  be  used  where  appropriate 
with  different  children  with  very  different  needs. 
Ultimately,  the  children  benefit  if  there  are  a 
range  of  stimuli  and  contexts  in  which  learning 
takes  place  and  that  should  be  ‘what’s  in  it  for 
them’! 
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REAL  TIME  CHANGING  VIRTUAL  EN¬ 
VIRONMENTS:  A  NEW  TOOL  FOR 
VIRTUAL  THERAPY 

M.  Alcaniz,  Ph.  D.,  Beatriz  Rey,  M.Sc.; 
Javier  Montesa,  M.Sc.;  Jose  Antonio 
Lozano,  Ph.D.;  M.  Carmen  Juan,  Ph.  D. 

Medical  Image  Computing  Laboratory 
(MedICLab),  Valencia,  Spain 

Research  Status;  Complete. 

Environments  used  up  to  now  for  therapeu¬ 
tic  applications  are  invariable  ones  whose 
content  cannot  be  changed  by  the  therapist 
or  by  the  patient.  However,  this  is  a  techni¬ 
cal  issue  that  can  be  solved  with  current 
technology.  In  this  paper,  we  describe  a 
virtual  environment  that  has  been  devel¬ 
oped  taking  into  account  this  factor.  The 
main  technical  feature  of  the  environment  is 
that  its  aspect  can  be  controlled  and  modi¬ 
fied  by  the  therapist  that  conducts  the  clini¬ 
cal  sessions  depending  on  the  emotions 
that  the  patient  is  feeling  at  each  moment. 
The  applications  of  these  dynamic  changes 
are  not  limited  to  the  field  of  clinical  psychol¬ 
ogy.  They  open  a  new  arena  of  possibilities 
for  many  other  kinds  of  applications  includ¬ 
ing  industry,  architecture,  medicine,  etc. 
The  virtual  environment  that  is  described  in 
this  paper  is  a  step  towards  a  truly  adaptive 
display. 
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THE  TREATMENT  OF  ARACHNOPHO- 
BIA 

Juan,  M.C.\  Botella,  C.  Banos,  R.^, 
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’  MedICLab  (Universidad  Politecnica  de  Valencia) 
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®  Universidad  de  Valencia 

Spider  phobia  could  be  a  great  problem  for 
people  who  suffer  from  this  kind  of  phobia 
and  they  have  to  visit  or  to  stay  in  places 
where  these  insects  could  appear.  Exposure 
therapies  and  Virtual  Reality  applications 
have  proved  to  be  effective,  but  till  now  Aug¬ 
mented  Reality  (AR)  has  not  been  used  in 
this  kind  of  treatments.  We  have  deveioped 
the  first  AR  system  for  the  treatment  of 
arachnophobia.  Our  AR  system  was  devel¬ 
oped  using  ARTooiKit  software.  We  have 
modelied  three  different  types  of  spiders. 
These  spiders  move  their  legs.  Both  the 
model  and  the  basic  movements  have  been 
modeiied  using  3DStudio  Max.  Textures  are 
created  using  Adobe  Photoshop  7.0.  Objects 
have  been  exported  to  VRML  format.  GLUl 
Library  has  been  used  to  develop  the 
Graphic  User  Interface.  To  include  sounds, 
OpenAL  Library  has  been  used.  When  the 
patient  or  the  therapist  kills  a  spider  using  a 
flyswatter,  the  system  plays  a  squishing 
sound  like  when  you  kill  a  real  spider.  If  the 
patient  or  the  therapist  uses  an  insecticide, 
the  system  plays  the  sound  of  a  spray  can 
like  when  you  use  a  real  insecticide.  The 
user  can  choose  that  one  or  more  spiders 
appear/disappear.  Spiders  can  appear/ 
disappear  in  increments  of  3  or  20.  When 
more  spiders  have  to  appear,  they  appear 
randomly  and  half  of  them  are  close  to  the 
center  and  they  can  move  towards  the  out¬ 
side  of  the  image.  The  rest  of  the  spiders  are 
far  away  as  possible  from  the  center  and 
they  can  move  towards  the  center.  Four  pa¬ 
tients  have  been  treated  with  the  system. 
They  met  DSM-IV  criteria  for  phobia  to  smaii 
animals  specifically,  fear  of  spiders.  Three  of 
them  were  females  and  one  male.  The  expo¬ 
sure  directives  of  Ost  were  followed  in  order 
to  carry  out  the  exposure  session.  The  ses¬ 
sions  were  about  one  hour.  Subjective  units 
of  discomfort  scale  (SUDS)  (0=no  anxiety. 
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10=high  anxiety)  was  used  to  measure  the 
degree  of  anxiety  during  the  AR  exposure. 
The  anxiety  level  in  some  moment  of  the  ex¬ 
posure  sessions  arrived  to  10  and  decreased 
till  0.  So,  the  system  stimulated  anxiety  level 
of  patients.  Before  the  exposure  session,  pa¬ 
tients  had  to  enter  to  a  room  where  a  spider 
was  inside  a  box.  None  of  them  were  able  to 
interact  with  the  alive  spider.  After  the  AR 
session,  patients  were  able  to  interact  and  to 
kill  by  themselves  real  spiders.  So,  our  sys¬ 
tem  has  helped  four  people  to  overcome  their 
spider  phobia  in  one  hour  of  AR  exposure 
session.  We  are  going  to  test  our  system  with 
more  patients.  Patients  also  answered  three 
questions  related  to  the  degree  of  presence 
experienced  in  the  AR  session.  All  patients 
scored  more  than  7  (0-10  scale)  in  all  ques¬ 
tions.  These  scores  reflect  the  high  sense  of 
presence  and  reality  judgment  that  the  pa¬ 
tients  experienced.  This  first  system  and  its 
first  results  demonstrate  that  AR  can  be  suc¬ 
cessfully  applied  to  psychological  treatments. 
We  hope  that  new  AR  applications  would  be 
developed  in  future  to  treat  other  kind  of  pho¬ 
bias. 
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University  of  Quebec  in  Outaouais  Cyberpsychol¬ 
ogy  Lab 

Cognitive-behavior  therapy  is  a  well  validated 
psychologicai  treatment  for  anxiety  disorders. 
However,  less  than  15%  of  people  suffering 
from  an  anxiety  disorder  receive  CBT  or  effec¬ 
tive  medication.  One  possible  explanation  is 
that  patients  don’t  have  access  to  CBT  thera¬ 


pists.  Videoconferencing  could  be  use  to 
increase  the  accessibility  of  validated  treat¬ 
ment  (i.e.,  CBT)  to  reach  patients.  It  is  im¬ 
portant  to  determine  is  videoconferencing 
can  be  efficient  to  give  a  psychological  treat¬ 
ment.  The  aim  of  the  study  is  to  compare  the 
effectiveness  of  a  validated  treatment  deliv¬ 
ered  through  videoconferencing  and  face-to- 
face. 

Method/Tools 

For  the  study,  30  participants  (15  =  condition 
face-to-face;  15  =  videoconferencing)  were 
SCID-diagnosed  in  a  face-to-face  interviews. 
All  participants  were  referrals  to  a  mental  clinic 
in  Montreal  (remote  site  180  km  distance  from 
the  local  site)  or  in  Gatineau  (local  site).  The 
selection  criteria  were:  1)  principal  diagnosis 
of  panic  disorder  with  agoraphobia;  2)  dura¬ 
tion  of  illness  for  more  than  6  months;  3)  not 
receiving  concurrent  psychotherapy;  4)  no 
comorbid  diagnosis  invalidating  the  study.  All 
participants  received  12  individual  weekly  ses¬ 
sions  of  cognitive-behavior  therapy.  The  effi¬ 
cacy  measures  were  completed  at  pre-  and 
post-treatment  and  included  the  Panic  and 
Agoraphobia  Scales  and  the  Agoraphobic 
Cognitions  Questionnaire. 

Results 

For  all  the  measures,  the  results  demon¬ 
strated  significant  improvement  (p<.05)  for  all 
the  participants  in  both  conditions  (face-to- 
face  and  videoconferencing). 

Conclusion 

These  results  of  the  study  show  the  effective¬ 
ness  of  the  cognitive-behavioral  therapy  de¬ 
livered  through  face-to-face  and  via  video- 
conferencing.  Those  results  may  have  an 
impact  on  the  availability  and  the  distribution 
of  mental  health  services  for  panic  disorder 
with  agoraphobia  and  other  anxiety  disor¬ 
ders. 
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NON-CONVENTIONAL  E-MENTAL 
HEALTH  SYSTEM  TO  SUPPORT  SER¬ 
VICES  TO  REMOTE  AREAS 

Dr.  Angelos  Amditis\  Ms.  Zoi  Lentziou\  Dr. 
Euangelos  Bekiaris^,  Dr.  Maria  Fernanda 
Cabrera^,  Dr.  Alex  Bullinger'* 

'  Institute  of  Communication  and  Computer  Sys¬ 
tems  (ICCS) 

^Center  for  Research  and  Technology  Hellas/  Hel¬ 
lenic  Institute  of  Transport  (CERTH/HIT) 
^Telecommunication  Engineering  School,  Techni¬ 
cal  University  of  Madrid  (UPM) 

''Center  of  Applied  Technologies  in  Mental  Health 
(COAT) 

This  paper  is  the  output  of  the  work  performed 
in  all  multidisciplinary  tasks  of  the  WP,  while 
focusing  on  defining  the  technical  specifica¬ 
tions  and  operative  scenarios  for  the  IS¬ 
LANDS  system  in  order  to  develop  services  to 
provide  modular,  non-conventional,  remote 
psychiatric  and  psychotherapeutic  assistance 
for  remote  areas. 

The  ISLANDS  project  basically  aims  at: 

•  Improving  the  quality  of  life  of  the  users, 

•  Enhancing  the  quality  of  mental  health 
care  and  the  economic  strength  of  the 
region 

•  Overweighting  the  costs  of  implementa¬ 
tion  and  service  support. 

•  Reducing  inequalities  in  mental  health 
services  and  status  among  European  re¬ 
gions. 

Building  a  complicated  system  like  the  IS¬ 
LANDS  system  was  a  time  consuming  and 
complicated  task,  since  the  combination  of 
both  the  delivery  of  three  different  services 
(i.e.  screening,  counselling  and  therapy)  to 
three  different  target  groups  (i.e.  general  prac¬ 
titioners,  patients  and  their  carers)  and  the 
compatibility  between  the  standards  and  the 
situation  that  exists  in  three  different  Euro¬ 
pean  areas  (Spain,  Greece  and  France)  is 
required.  This  is  the  reason  why  a  thorough 
analysis  regarding  the  specifications  and  the 
requirements  of  the  various  components  and 
the  peripheral  devices  of  the  overall  system 
has  taken  place  and  an  elaborate  study  for 
the  design  of  the  system  architecture  has 
been  carried  out. 

Primarily,  the  architecture  of  the  ISLANDS 
system  takes  the  following  parameters  into 
account: 

•  The  various  current  technical  ways  of  es¬ 


tablishing  e-mental  health  and  their  re¬ 
quirements  in  relation  to  the  existing 
equipment  in  the  area  of  telemedicine  in 
general. 

•  Cost  efficiency  issues. 

•  Liability. 

•  Risk  analysis 

The  User  (i.e.  General  Practitioner,  Patient 
and/or  relative)  can  have  access  through  the 
network  (i.e.  Internet)  to  the  ISLANDS  Multi- 
Access  Center.  The  IMAC  has  the  potentials 
to  allow  users  to  access  the  remote  services 
whatever  access  terminal  or  combination  of 
terminals  they  may  choose  to  use. 

IMAC  consists  of  four  parts  mainly: 

♦  the  ISLANDS  database 

♦  the  ISLANDS  router 

♦  the  ISLANDS  tools  and 

♦  the  Knowledge  Management  Organ¬ 
iser 

It  is  self  evident  that  in  most  of  the  cases  the 
data  transmission  urgency  should  be  taken 
into  consideration.  This  means  that  since  the 
service  delivery  frequency  is  dependent  on 
the  type  of  the  content,  the  format  and  the 
importance  of  the  transmitted  medical  data 
were  identified  and  therefore  the  technology 
that  should  be  used  was  defined  to  cover  the 
application  needs.  For  instance,  screening  a 
patient,  which  can  be  considered  as  a  low 
importance  service  delivery,  can  be  sup¬ 
ported  by  text.  The  technology  that  can  be 
used  in  that  case  is  possibly  the  mail.  On  the 
other  hand,  providing  guided  therapy  to  the 
patient,  which  is  a  case  of  high  significance 
and  importance,  require  the  transmission  of 
video  (audio  and  visual  contact  with  the  doc¬ 
tor).  This  leads  to  the  conclusion  that  the 
technology  that  could  be  used  to  apply  this 
situation  is  Videoconference  or  Computer 
conference. 
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’  Universitat  Jaume  I  Castellon.  (Spain). 

^  Universidad  de  Valencia  (Spain). 

During  the  last  decade  notable  advances  in 
the  psychological  treatments  field  have  been 
produced.  At  this  moment  an  important 
amount  of  effective  treatments  or  “based  on 
evidence”  treatments  {Task  Force  on  Promo¬ 
tion  and  Dissemination  of  Psychological  Pro¬ 
cedures,  1995)  are  available.  The  cognitive 
model  has  had  a  notable  contribution  for  this 
important  advance  to  be  achieved,  however, 
from  a  therapy  perspective,  this  model  is  still 
far  away  from  understanding  and  controlling 
the  human  change  processes.  These  insuffi¬ 
ciencies  come  from  the  theoretical  shortcom¬ 
ings,  that  is,  there  exist  discrepancies  be¬ 
tween  the  lineal  logic  of  many  of  the  cognitive 
perspectives  and  the  complexity  of  human 
functioning.  Within  these  insufficiencies  is 
worth  to  point  out  the  scarce  attention  that 
has  been  paid  to  emotions  in  therapy  for  a 
long  time. 

One  of  the  objectives  of  the  European  project 
EMMA  {Engaging  Media  for  Mental  Health)  is 
to  examine  the  possibility  of  using  new  tech¬ 
nologies  (virtual  reality)  with  the  aim  of  treat¬ 
ing  emotions  from  a  different  perspective  than 
the  traditional  one.  In  the  EMMA  project  we 
have  developed  a  series  of  emotional  devices 
which  can  be  customized  so  they  can  be  full 
of  meaning  and  contain  the  fundamental  ele¬ 
ments  that  the  person  has  to  cope  with.  That 
is,  our  objective  is  to  try  to  show  in  a  physical 
way  the  personal  meanings  and  the  emotions 
associated  to  those  meanings  and  to  examine 
to  which  extent  this  strategy  helps  the  person 
in  the  process  of  change.  The  purpose  is  to 
create  a  series  of  “experiences”  in  therapy 
that  can  be  used  to  activate,  correct,  struc¬ 
ture,  and  restructure  previous  life  experiences 
that  can  serve  as  cognitive-emotional  struc¬ 
tural  frames  from  which  a  new  way  of  proc¬ 
essing  and  integrating  present,  past,  and  fu¬ 
ture  experiences  can  be  structured.  In  this 
work  we  present  the  EMMA  environment  and 
the  clinical  treatment  protocol  for  pathological 
grief  in  a  case  study. 
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Presence  and  Reality  Judgment  are  two  im¬ 
portant  variables  to  take  into  account  in  the 
Virtual  Reality  (VR)  field.  So  far,  scientific 
literature  has  paid  attention  to  the  construct 
of  presence,  and  several  assessment  meas¬ 
ures  are  available.  However,  the  concept  of 
Reality  Judgment  has  received  less  attention, 
and  usually,  it  has  been  subsumed  into  the 
concept  of  presence.  No  too  much  effort  has 
been  dedicated  to  test  whether  or  not  both 
constructs  refer  to  the  same  domain.  In  a 
previous  work,  our  team  developed  a  78-item 
self-report  questionnaire  that  assesses  both 
constructs:  sense  of  presence  and  reality 
judgment  (Banos  et  al.,  2000).  This  was  ap¬ 
plied  to  124  undergraduate  students  from  the 
University  Jaume  I  of  Castelibn  (Spain)  and 
the  University  of  Washington  (Seattle).  A 
three  factor  solution  was  obtained:  a  reality 
judgment  factor,  an  interaction/external  cor¬ 
respondence  factor  and  an  attention/ 
absorption  factor.  While  the  reality  judgement 
items  grouped  in  a  single  factor,  items  con¬ 
cerning  presence  were  watered  down  among 
the  three  obtained  factors.  We  believe  that 
the  sample  employed  had  influenced  re¬ 
markably  on  the  results.  The  scenarios 
where  the  participants  were  immersed  did 
not  stimulate  any  type  of  emotion  (those  par¬ 
ticipants  that  could  have  had  the  slightest 
problem  in  any  of  the  environments  were 
excluded),  and  this  fact  prevented  any  emo¬ 
tional  involvement  or  internal  correspon¬ 
dence  factors  from  showing  up.  The  following 
task  was  to  apply  the  questionnaire  to  clinical 
samples  to  elucidate  whether  the  same  or 
different  factors  emerged.  Preliminary  results 
obtained  with  clinical  and  subclinical  partici¬ 
pants  belonging  also  to  Spanish  and  North 
American  samples  showed  that  in  VR  appli¬ 
cations  for  Clinical  Psychology  emotion  was 
playing  an  important  role  in  the  sense  of 
presence  and  the  reality  attribution  of  users 
(Banos  et  al.,  2001).  In  this  case,  the  virtual 
environments  were  able  to  elicit  emotions  in 
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the  participants,  so  items  related  to  emotions 
and  sensations  were  the  most  important. 
However,  an  important  shortcoming  of  this 
study  was  the  small  size  of  the  sample 
(N=112).  Therefore,  the  aim  of  the  present 
work  is  to  apply  the  Presence  and  Reality 
Judgement  Questionnaire  to  a  larger  clinical 
and  subclinical  sample. 
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FACTORS  RELATED  TO  SUCCESS  OF 
HELPING  HIGHLY  DISTRESSED  INDI¬ 
VIDUALS  THROUGH  EMOTIONAL 
SUPPORT  ONLINE  CHAT 

Azy  Barak,  PhD 
University  of  Haifa 

The  purpose  of  the  study  was  to  examine  the 
contribution  made  by  dimensions  of  session 
impact  (experts'  evaluation  of  session  proc¬ 
ess,  that  includes  factors  of  depth  and 
smoothness,  and  experts'  evaluation  of  post¬ 
session  factors  of  client's  mood,  that  includes 
positivity  and  emotional  arousal)  and  of  sev¬ 
eral  textual  variables  (positive  and  negative 
emotional  expressions;  helper's  and  client's 
writing  length)  to  the  success  of  emotional 
support  conversations  carried  out  by  trained, 
paraprofessional  helpers  on  an  Internet  chat 
with  highly  distressed  individuals.  Two  studies 
were  conducted  at  SAHAR,  an  exclusively 


online,  Israeli  emotional  support  service  for 
suicidal  and  highly  distressed  people  who 
had  experienced  various  negative  conditions, 
including  those  related  to  rape  and  sexual 
abuse,  eating  disorders,  depression,  social 
anxiety,  drinking  and  drug  use,  homosexual¬ 
ity,  and  domestic  violence.  Study  1  compared 
40  successful  conversations  (as  indicated  by 
clients  at  the  termination  stage  of  session) 
executed  through  instant  messaging  and 
chat  tools  with  40  other  conversations  (not 
indicated  as  successful  by  clients),  using  ex¬ 
pert  Judgments,  on  session-impact  dimen¬ 
sions  as  well  as  objective  word  count  for  tex¬ 
tual  variables.  Study  2  examined  correlations 
between  helpers'  evaluation  of  the  sessions' 
contribution  to  clients  in  60  (other)  support 
conversations  and  session  impact  factors 
and  textual  dimensions.  The  findings  of 
Study  1  showed  that  all  four  impact  dimen¬ 
sions,  as  well  as  expression  of  negative  emo¬ 
tions,  significantly  differentiated  between  suc¬ 
cessful  and  unsuccessful  conversations.  In 
Study  2,  the  results  showed  that  all  four  im¬ 
pact  dimensions  positively  correlated  with 
helpers'  evaluations  (yielding  multiple  R=.54), 
as  well  as  the  length  of  helper's  and  client's 
writing.  The  implication  of  these  studies  are 
that  similar  to  offline  counselling  sessions, 
deep,  smooth  conversations  that  yield  posi¬ 
tive  responses  and  arouse  clients'  emotions 
in  online  support  are  more  successful  than 
shallow,  bumbling  conversations  that  leave 
clients  emotionally  indifferent.  Ventilating 
negative  feelings  and  more  expression  in 
writing,  by  both  helpers  and  clients,  seem  to 
be  important  factors,  as  well. 
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University  of  Transylvania  Brasov 

In  this  paper  we  propose  an  improvement 
method  of  audio  video  techniques  and  smart 
environment  for  optimization  and  objective 
methods  of  investigation  in  the  bio-systems 
and  for  the  disadvantage  people  assisting. 
Taking  in  account  the  necessity  for  develop¬ 
ment,  the  new  facilities  and  services  it  is  pro¬ 
posed  the  accomplish  a  modulated  infrastruc¬ 
ture  which  can  generate  the  optimum  systems 
for  different  investigations  of  the  health  prob¬ 
lems,  the  creation  of  the  smart  environments 
for  the  medical  assistance,  the  generation  of 
new  methods-virtual  reality-  to  prevent  the 
deficiencies  installation  to  the  children  and 
adults.  The  most  important  objectives  of  the 
researches  are:  the  implementation  of  this 
health  investigation,  assisting  and  information 
structure  into  public  service  by  e-Health  sys¬ 
tems,  the  evaluation  of  the  research  impact 
on  the  assurance  of  the  compatibilities  de¬ 
mands  for  Romania  integration  in  European 
system,  and  also  the  using  of  the  methods 
like  virtual  reality  and  biotelemetry,  for  devel¬ 
oping  of  the  news  way  to  assist  the  people. 
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THE  TREATMENT  OF  EMOTIONS  IN  A 
VIRTUAL  WORLD.  APPLICATION  IN  A 
CASE  OF  POSTTRAUMATIC  STRESS 
DISORDER 
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During  the  last  decade  notable  advances  in 
the  psychological  treatments  field  have  been 
produced.  At  this  moment  an  important 
amount  of  effective  treatments  or  “based  on 
evidence”  treatments  {Task  Force  on  Promo¬ 
tion  and  Dissemination  of  Psychoiogical  Pro¬ 
cedures,  1995)  are  available.  The  cognitive 
model  has  had  a  notable  contribution  for  this 


important  advance  to  be  achieved,  however, 
from  a  therapy  perspective;  this  model  is  still 
far  away  from  understanding  and  controlling 
the  human  change  processes.  These  insuffi¬ 
ciencies  come  from  the  theoretical  shortcom¬ 
ings,  that  is,  there  exist  discrepancies  be¬ 
tween  the  lineal  logic  of  many  of  the  cognitive 
perspectives  and  the  complexity  of  human 
functioning.  Within  these  insufficiencies  it  is 
worth  to  point  out  the  scarce  attention  that 
has  been  paid  to  emotions  in  therapy  for  a 
long  time. 

One  of  the  objectives  of  the  European  project 
EMMA  {Engaging  Media  for  Mental  Health)  is 
to  examine  the  possibility  of  using  new  tech¬ 
nologies  (virtual  reality)  with  the  aim  of  treat¬ 
ing  emotions  from  a  different  perspective 
than  the  traditional  one.  In  the  EMMA  project 
we  have  developed  a  series  of  emotional 
devices  which  can  be  customized  so  they 
can  be  full  of  meaning  and  contain  the  funda¬ 
mental  elements  that  the  person  has  to  cope 
with.  That  is,  our  objective  is  to  try  to  show  in 
a  physical  way  the  personal  meanings  and 
the  emotions  associated  to  those  meanings 
and  to  examine  to  which  extent  this  strategy 
helps  the  person  in  the  process  of  change. 
The  purpose  is  to  create  a  series  of 
“experiences”  in  therapy  that  can  be  used  to 
activate,  correct,  structure,  and  restructure 
previous  life  experiences  that  can  serve  as 
cognitive-emotional  structural  frames  from 
which  a  new  way  of  processing  and  integrat¬ 
ing  present,  past,  and  future  experiences  can 
be  structured.  In  this  work  we  present  the 
EMMA  environment  and  the  clinical  treat¬ 
ment  protocol  for  posttraumatic  stress  disor¬ 
der  in  a  case  study. 
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MIXING  REALITIES?  AN  AUGMENTED 
REALITY  SYSTEM  FOR  THE  TREAT¬ 
MENT  OF  SPIDERS  AND  COCK¬ 
ROACHES  PHOBIA 
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Augmented  Reality  (AR)  consists  of  introduc¬ 
ing  virtual  elements  in  the  real  world.  That  is, 
the  person  is  seeing  an  image  conformed  by 
a  visualization  of  the  real  world  and  by  a  se¬ 
ries  of  virtual  elements  that  are  over-imposed 
in  the  real  world.  The  main  aspect  of  AR  is 
that  virtual  elements  provide  relevant  and 
useful  information  to  the  person  that  does  not 
exist  in  the  real  world.  AR  has  a  high  potential 
and  has  been  already  used  in  several  fields: 
medicine,  army,  training,  engineering,  design, 
robotic,  etc.  Up  to  now  AR  has  not  been  used 
in  the  psychological  treatments  area.  How¬ 
ever,  AR  presents  several  advantages.  Like  in 
the  classic  virtual  reality  systems,  it  is  possi¬ 
ble  to  have  a  total  control  over  the  virtual  ele¬ 
ments  over-imposed  in  the  real  world  and  how 
to  interact  with  them.  But,  AR  has  and  addi¬ 
tional  advantage;  it  facilitates  the  person’s 
sense  of  presence  (the  sense  of  being  there) 
and  reality  judgment  (to  judge  the  experience 
as  real).  This  is  possible  because  the  environ¬ 
ment  where  the  person  is  placed  and  what  the 
person  is  seeing  are,  in  fact,  “the  reality".  In 
this  work  the  data  of  a  series  of  case  studies 
in  which  AR  has  been  used  for  the  treatment 
of  specific  phobia  (spiders  and  cockroaches 
phobia)  are  presented.  Results  are  very  prom¬ 
ising.  The  patients  showed  a  high  degree  of 
fear  and  avoidance  in  the  behavioural  avoid¬ 
ance  test  at  pre-test  and,  these  scores  not 
only  decreased  after  treatment,  but  the  pa¬ 
tients  were  able  to  approach,  interact  and  kill 
alive  insects. 
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Measuring  the  subjective  feeling  of  presence  in 
virtual  environments  usually  relies  on  question¬ 
naires.  These  instruments  are  long  and,  while 
many  items  are  useful  to  tap  the  diverse  subtleties 
and  dimensions  of  a  constract,  it  poses  a  signifi¬ 
cant  challenge  if  a  researcher  wants  to  measure 
presence  during  a  VR  immersion.  This  presenta¬ 
tion  will  document  the  validation  process  of  a 
single-item  measure  of  presence.  The  question 
«To  which  extend  do  you  feel  present  in  the  vir¬ 
tual  environment,  as  if  you  were  really  there»  was 
subjected  to:  a  content  and  face  validity  study, 
two  test-retest  reliability  studies,  a  convergent 
and  divergent  validity  study,  and  two  sensitivity 
studies. 

Content  and  face  validity 

The  goal  of  the  first  study  was  to  ascertain 
that  the  single-item  presence  is  well  under¬ 
stood  by  the  general  population.  A  commu¬ 
nity  sample  of  49  adults  (mostly  low  to  mod¬ 
erate  socio-economic  status  and  not  highly 
educated)  were  immersed  for  7  minutes  in  a 
virtual  environment  in  a  shopping  mall.  After 
the  immersion,  participants  rated  on  a  scale 
from  0  to  10  how  well  they  understood  the 
meaning  of  the  item,  as  well  as  the  meaning 
of  control  items  (including  items  from  other 
popular  presence  measure).  The  results 
showed  that  the  question  was  very  well  un¬ 
derstood.  The  clarity  if  the  question  was  as 
good  as  controlled  items  drawn  from  a  psy¬ 
chological  test  validated  for  adults  with  the 
reading  ability  of  a  grade-five  population.  The 
clarity  was  statistically  better  than  the  control 
items  that  were  designed  to  be  difficult  to 
understand  (e.g.,  items  with  double¬ 
negatives). 

Test-retest 

A  sample  of  31  university  students  completed 
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the  presence  question  on  two  occasions  dur¬ 
ing  a  7  minutes  VR  immersion,  and  another 
sample  of  26  adults  completed  the  presence 
question  after  two  different  VR  immersions. 
The  test-retest  coefficients  were  very  high 
(.21  and  .83,  respectively). 

Convergent  and  divergent  validity 

The  single-item  measure  of  presence  corre¬ 
lates  significantly  with  the  Presence  Question¬ 
naire,  and  much  less  with  Perceived  Realism 
and  the  Immersive  Tendencies  Questionnaire. 

Sensitivity 

Two  studies  were  devised  to  assess  the  sen¬ 
sitivity  of  the  item  to  experimental  manipula¬ 
tions.  In  the  first  study,  33  participants  were 
immersed  twice  in  the  same  virtual  under  a 
condition  that  maximise  presence  and  under  a 
condition  that  hinder  presence.  In  a  second 
study,  29  snake  phobics  were  subjected  to 
psychological  manipulations  where  anxiety 
was  induced  or  not  by  leading  them  to  believe 
that  the  virtual  environment  could  contain 
dangerous  hidden  snakes.  In  both  studies,  the 
repeated  measures  ANOVA  confirmed  the 
sensitivity  of  our  measure. 

Conclusion  and  novelty 

Item-response  theory  would  recommend  to  be 
careful  in  using  only  one  item  to  measure  a 
construct,  while  practical  factors  such  as  dis¬ 
traction  would  militates  in  favour  of  using  the 
least  intrusive  measure  as  possible.  The  vali¬ 
dation  process  confirmed  that  a  single-item 
measure  of  the  subjective  feeling  of  presence 
is  well  understood,  reliable  and  valid.  These 
results  will  be  very  useful  for  any  researcher 
interested  in  measuring  presence  while  users 
are  immersed  in  a  virtual  environment.  The 
presentation  of  the  results  of  each  study 
should  also  be  of  interest  to  the  audience. 
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Background 

In  child  and  adolescent  therapy,  multimedia 
is  most  commonly  used  where  the  child  and 
therapist  use  professionally  made  CD-ROM’s 
of  interactive  stories,  computer  games,  or 
internet  resources  to  access  information  and 
facilitate  communication.  This  involves  listen¬ 
ing,  watching,  and  reading.  The  interactive 
elements  allow  users  a  degree  of  control 
over  the  narrative  and  direction  of  a  story. 
This  is  mainly  passive.  “Transforming  Sto¬ 
ries”  takes  it  a  step  further  by  using  multime¬ 
dia  in  therapy  in  a  constructive  way,  allowing 
the  child  to  build  their  own  story.  In  his  re¬ 
search  in  the  field  of  education  Papert  (1996) 
highlighted  the  importance  of  children  making 
their  own  artifacts,  something  that  is  a  real 
reaction  to  their  world.  This  premise  can  also 
be  applied  to  the  therapeutic  process. 

Method/Tools 

Transforming  Stories  is  a  database  driven 
multimedia  storytelling  tool  delivered  both 
online  and  on  stand  alone  platform.  The  soft¬ 
ware  currently  being  developed  will  allow  the 
users  to  choose  a  story  template  and  build 
up  the  story  plot  using  customisable  charac¬ 
ters  and  background  scenes,  adding  in  their 
own  voiceover  and  animation.  The  authoring 
timeline  is  presented  in  an  immediate  way 
using  a  drag  and  drop  facility  for  placing  story 
objects  and  further  features  such  as  position¬ 
ing  tools,  playback  and  story  editing.  There  is 
also  a  text  tool  allowing  them  to  integrate 
their  own  captions.  Other  options  include 
building  from  seed  stories  where  they  can 
change  the  plotline  and  add  in  their  own 
characters  and  dialogue.  Finally  they  can 
upload  and  share  their  stories  to  an  online 
community. 

In  the  method,  the  therapist  introduces  the 
idea  of  making  a  story  and  shows  examples 
of  what  is  possible  using  the  software.  It  can 
be  a  story  tailored  to  a  problem  that  needs 
resolution  or  a  general  story  that  brings  about 
some  positive  change  in  the  storyline.  A  key 
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therapeutic  feature  is  a  notebook  facility 
where  the  child  and  therapist  keep  a  log  of 
their  thoughts  on  the  process  and  their  analy¬ 
sis  of  the  story.  This  allows  for  reflection  and 
gives  the  therapist  the  opportunity  to  prompt 
new  ideas  for  problem  solving. 

Evaluation 

The  current  research  focuses  on  evaluating 
the  therapeutic  use  of  multimedia  authoring 
tools  now  being  widely  used  in  the  classroom 
setting.  This  research  is  based  on  case  study 
examination  of  the  benefits  of  using  these 
tools  in  engaging  adolescents  and  progress¬ 
ing  the  therapeutic  relationship  in  a  small 
sample  of  adolescents  attending  the  Mater 
Child  and  Adolescent  Psychiatry  Service, 
Dublin,  Ireland. 

The  second  stage  of  the  research  will  involve 
the  initial  testing  of  “Transforming  Stories” 
across  a  range  of  client  groups  and  profes¬ 
sional  disciplines  including  Psychotherapy, 
Social  Work,  Psychology,  and  Psychiatry. 
This  testing  will  involve  a  qualitative  study. 

Novelty/Discussion 

“Transforming  Stories”  is  an  innovative  new 
approach  to  eliciting  stories  in  child  and  ado¬ 
lescent  therapy.  Over  time  the  online  forum 
will  provide  an  expanding  database  of  story 
building  material  to  be  drawn  from.  The 
planned  integration  of  drawing  tools  and  other 
authoring  features  will  give  increase  the  crea¬ 
tive  dimension  and  allow  the  users  to  create 
uniquely  designed  work. 
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PEOPLE  DEALING  WITH  DEPRES¬ 
SION  AND  OTHER  MENTAL  HEALTH 
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It  is  widely  accepted  that  children  and  young 
people  enjoy  using  computers.  Seymour  Pa¬ 
pert  (1996)  suggests  that  ‘it  is  timely  for 
school  counsellors  and  child  therapists  to 
bridge  the  digital  generation  gap  and  inno¬ 
vate  with  computers  in  their  work  with  chil¬ 
dren  of  the  digital  age.’  Working  Things  Out 
(WTO)  was  developed  in  response  to  the 
growing  need  for  computer-based  resources 
for  adolescents  in  therapy.  It  is  an  interactive 
CD  ROM/DVD  developed  as  a  means  of  en¬ 
gaging  adolescents  about  mental  health  is¬ 
sues  by  giving  them  information  in  the  form 
of  animated  personal  stories  told  by  other 
young  people  dealing  with  problems  such  as 
depression,  bullying,  eating  problems,  and 
self-harm.  The  CD-ROM/DVD  is  accompa¬ 
nied  by  a  manual  for  professionals  on  how  to 
use  the  resource. 

With  the  help  of  therapists,  and  in  collabora¬ 
tion  with  graphic  designers,  animators,  and 
multimedia  professionals,  the  11  young  peo¬ 
ple  who  participated  in  the  project  told  their 
stories,  narrated  in  their  own  voice,  and  illus¬ 
trated  by  graphics  and  animation.  ‘Working 
Things  Out’  is  currently  being  used  as  an 
educational  and  therapeutic  tool  with  other 
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adolescents  at  risk  of  mental  health  problems, 
both  as  a  way  of  engaging  young  people  to 
reflect  about  mental  health  issues  and  as  a 
means  of  inviting  them  to  tell  their  own  story. 
The  paper  describes  the  background  and  de¬ 
velopment  of  the  'Working  Things  Out'  project, 
including  samples  from  the  stories  and  a  de¬ 
scription  of  how  the  CD-ROM/DVD  can  be 
used  in  psychotherapy. 
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This  paper  deals  with  the  ongoing  develop¬ 
ments  towards  immersive  3-D  displays  that 
require  no  special  eyewear  like  shutter 
glasses  anymore.  Once  these  devices  will 
have  penetrated  the  market  to  a  larger  scale 
they  could  easily  mark  the  end  to  the  era  of 
huge  rear  projection  -  based  installations 
such  as  Powerwalls  or  CAVE  -  systems.  At 
present  only  a  handful  of  these  3-D  display 
devices  are  on  the  market,  most  of  them  still 
promising  more  than  what  they  are  able  to 
fulfill  in  the  end.  But  technology  development 
in  this  sector  is  moving  with  utmost  velocity 
and  ferocity.  By  the  end  of  2005  marketing  of 


desktop  3-D  displays  will  begin  in  earnest 
opening  up  a  vast  number  of  potential  appli¬ 
cations  not  only  for  scientist,  engineers  and 
IT  specialists  but  literally  for  everybody. 
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Specific  phobias  are  one  of  the  most  frequent 
mental  health  problems  and  can  lead  to 
years  of  personal  suffering.  The  most  effec¬ 
tive  treatment  is  exposure  therapy.  Our  aim 
was  to  proof  the  feasibility  and  efficacy  of 
virtual  environments  in  treating  acrophobia 
patients  using  a  manually  guided  exposure 
therapy.  Our  pilot  study  was  designed  as  a 
crossover  intervention  with  a  waiting  list  con¬ 
dition  as  a  control  group.  After  treatment,  our 
results  show  that  exposure  in  virtual  environ¬ 
ments  is  a  feasible  technique  can  provoke 
anxiety,  and  leads  to  a  therapeutic  effect. 
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It  has  been  already  accepted  that  the  im¬ 
provement  in  medical  care  cannot  be  made 
merely  by  adding  professionals  to  healthcare 
systems,  but  in  addition  require  restructuring 
the  patterns  of  health-care  delivery.  The  incor¬ 
poration  into  medical  practice  of  relevant  tech¬ 
nological  innovations  is  an  essential  part  of 
this  reorganisation.  This  innovation  is  produc¬ 
ing  a  promising  field  -  e-mental  health  -  whose 
focus  is  the  use  of  communication  and  infor¬ 
mation  technologies  to  improve  the  mental 
health  care  processes.  This  area  has  devel¬ 
oped  rapidly,  accumulating  knowledge  and 
proposing  innovative  affirmations.  This  is  a 
multidisciplinary  field  that  requires  the  coop¬ 
eration  among  different  professionals.  The 
results  of  this  collaboration  are  represented 
by  the  ISLANDS  project,  an  attempt  to  de¬ 
velop  and  comprehend  the  potential  of  e- 
mental  health,  which  specific  goal  was  to  de¬ 
velop  services  to  provide  modular,  non- 
conventional,  remote  psychiatric  and  psycho¬ 
therapeutic  assistance  for  remote  areas. 

The  project  started  with  a  literature  review  of 
the  state  of  the  art  on  remote  therapeutic  psy¬ 
chiatric  and  psychotherapeutic  interventions, 
complemented  by  field  work  with  the  realiza¬ 
tion  of  questionnaires  to  patients,  families  and 
doctors,  and  an  international  workshop,  to 
result  In  appropriate  service  delivery  scenar¬ 
ios. 

The  scenarios  specified,  the  different  user 
group  needs  and  the  epidemiological  findings 
led  to  the  definition  of  different  remote  service 
categories  (diagnosis,  counseling,  and  ther¬ 
apy)  for  patients,  informal  careers,  and  pro¬ 
fessionals,  as  well  as  an  overall  service  lay¬ 
out.  These  services  are  supported  by  interac¬ 
tive  and  user-friendly  tools  for  service  content 
presentation,  namely:  an  interactive  web  chat 
tool,  a  database  of  reference  case  studies,  an 
expert  tool  for  therapy  guidance,  a  tool  for 
service  confidentiality,  and  the  necessary 
communication  tools  and  service  delivery  plat¬ 
forms.  All  the  above  developments  are  inte¬ 
grated  into  modular  service  typologies,  taking 
into  account  relevant  security,  legal  and  ethi¬ 
cal  issues.  Four  typical  case  studies  of  psy¬ 
chological  problems  have  been  selected  that 
can  be  found  quite  often  in  normal  popula¬ 
tions  and  are  of  specific  interest:  post- 
traumatic  stress  disorder,  depression,  prob¬ 
lems  of  alcohol  abuse,  and  psychotic  disor¬ 
ders.  The  proposed  services  are  being  tested 
in  three  pilot  sites:  the  French  Overseas  De¬ 
partments,  the  Greek  Southern  Sporades, 


and  the  Spanish  western  Canary  Islands. 

Over  the  past  24  months,  the  ISLANDS  con¬ 
sortium  has  focused  in  the  development  and 
deployment  of  feasible  tools  for  diagnostic, 
counseling,  and  therapy  purposes.  According 
to  the  preliminary  results  of  a  questionnaire 
survey,  users'  acceptance  of  and  satisfaction 
with  the  technology  were  high.  Ninety  per¬ 
cent  of  the  patients  considered  that  they  re¬ 
ceived  the  follow-up  care  they  required. 

We  can  conclude  that  the  development  of  a 
knowledge-based  expert  tool  to  guide  the 
relevant  services  application  aims  to  avert 
erroneous  application  of  such  services,  by 
inexperienced  medical  personnel  or  the  us¬ 
ers  themselves  and  their  relatives. 

The  project  will,  it  is  hoped,  provide  leader¬ 
ship,  enhance  information  about  mental 
health  problems,  and  undertake  research  in 
cost-effective  policies  to  improve  the  mental 
disorders  addressed. 
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EMERGENCY  HEALTHCARE  APPLI¬ 
CATIONS 
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In  the  new  society,  mobility  has  become  a 
necessity  and  a  competitive  advantage,  both 
for  citizens  and  the  human  capital  of  compa¬ 
nies.  Its  next  step  is  clearly  to  become  a 
complete  product.  In  order  to  become  a  com¬ 
modity  for  European  society,  mobile  solutions 
must  provide  five  main  characteristics:  ubiq¬ 
uity,  transparency,  fundamentality,  universal¬ 
ity,  and  value  governance.  As  the  new  3G 
technology  is  "around  the  corner,"  suitable 
platforms  that  make  possible  an  interesting 
business  model  to  access  Pan-European 
mobile  services  is  a  "must"  that  hasn't  been 


227 


Annual  Review  of  CyberTherapy  and  Telemedicine 


resolved,  yet.  Facing  this  reality,  the  Euro¬ 
pean  Union  funded  XMOB,  an  1ST  project  that 
Joined  research  groups,  phone  operators, 
consultancy  companies,  industry,  and  emer¬ 
gency  medical  services  (EMS)  to  build  a  plat¬ 
form  aimed  to  host  mobile  applications.  The 
benefits  and  outputs  of  the  platform  were  vali¬ 
dated  through  the  development  of  a  vertical 
application  for  the  health  emergency  manage¬ 
ment. 

The  mobile  emergency  services  developed 
focused  on  the  following  main  challenges: 

Necessity  to  cover  a  broad  set  of  de¬ 
vices  and  channels 

Need  to  integrate  a  complex  range  of 
involved  actors  (e.g.,  insurance  call  centers, 
public  emergency  assistance,  patients'  histori¬ 
cal  medical  records  repositories,  hospitals, 
etc.) 

Provision  of  complete,  customized,  and 
timely  assistance  in  emergency  situations 
based  on  instant  and  secure  mobile  access  to 
standardized  patients  medical  records 

Existence  of  emergency  situations  that 
require  the  coordination  between  different 
countries  to  provide  an  accurate  attention  to 
the  patient 

The  methodological  approach  consisted  of 
two  steps;  to  build  a  reusable  horizontal  plat¬ 
form  and  then,  following  an  ASP  model,  to 
develop  vertical  mobile  applications  for  health 
emergency  management.  These  applications 
were  made  of  several  tools  that  help  the 
emergency  staff  and  the  other  actors  involved 
in  accessing  critical  data  of  patients. 

The  platform  was  validated  in  real  life  situa¬ 
tions  and  cost-effectiveness  and  patients' 
benefits  analysis  were  conducted.  The  trials 
were  carried  out  in  Madrid  and  Genoa  cities 
and  lasted  two  months. 

The  opinion  of  the  users  consisted  of  a  self- 
evaluation  with  a  questionnaire  distributed  to 
different  staff  categories.  The  questionnaires 
contained  closed-ended  questions  and  re¬ 
spondents  were  provided  space  to  add  written 
comments.  The  results  were  evaluated  in 
scores  on  two  scales,  the  "Usefulness"  scale 
denoting  potential  use  of  the  system,  and  the 
"Satisfying"  scale  reflecting  pleasantness.  The 
patient's  privacy  and  respect  issues  were  cov¬ 
ered. 

The  most  important  conclusion,  which  can  be 
derived  from  the  pilots'  comparisons,  is  the 


fact  that  all  user  groups  were  in  all  cases  and 
in  both  pilot  sites  positively  oriented  towards 
the  developed  system.  Nevertheless,  it 
should  be  remarked  that  all  EMS  do  not  need 
the  same  functionalities  in  their  pre-hospital 
emergency  management  due  to  specificity  of 
environments  (rural/urban,  medical/ 
paramedic)  so  that  the  main  achievement  of 
this  platform  is  to  provide  a  widely  open  inte¬ 
grated  solution.  This  will  enable  any  citizen  to 
benefit  from  the  same  quality  of  care,  any¬ 
where,  in  pre-hospital  emergencies  domain. 

The  benefits  of  a  seamless  infrastructure  are 
the  foundational  components  of  this  platform 
that  improve  the  efficiency  and  efficacy  of  the 
EMS.  This  translates  into  lives  saved  and 
morbidity  reduced. 
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GEDANKEN  EXPERIMENT  OF  HUMAN 
DYNAMICS 

Yang  Cai 

Carnegie  Mellon  University 

Understanding  human  dynamics  is  an  impor¬ 
tant  step  in  clinical  therapies,  such  as  blood 
glucose  management  or  weight  control.  It 
requires  insight  into  the  complex  interactions 
between  the  components  of  biomedical  sys¬ 
tems.  This  is  challenging  because  of  the 
enormous  amount  of  data  and  knowledge  in 
this  domain.  Gedanken  Experiment  is  an  in¬ 
tuitive,  qualitative  thinking  process,  which 
have  been  used  by  physicists  for  decades. 
Our  goal  is  to  enable  an  interactive  game-like 
environment  for  users  with  any  background, 
while  remaining  sufficiently  flexible  to  target 
medical  problems  at  a  level  of  abstraction, 
from  the  conformational  changes  of  a  protein 
to  the  interaction  of 
the  various  biochemical  pathways  in  our 
body.  Here,  we  present  an  interactive  and 
visual  problem-solving  environment  for  the 
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biomedical  domain.  We  designed  a  biological 
world  model,  in  which  users  can  explore  bio¬ 
logical  interactions  by  role-playing 
"characters"  such  as  cells  and  molecules  or 
as  an  observer  in  a  "shielded  vessel,"  both 
with  the  option  of  networked  collaboration  be¬ 
tween  simultaneous  users.  The  system  archi¬ 
tecture  of  these  "characters"  contains  four 
main  components:  (1)  Bio-behavior  is  mod¬ 
eled  using  cellular  automata.  (2)  Bio-morphing 
uses  vision-based  shape  tracking  techniques 
to  learn  from  recordings  of  real  biological  dy¬ 
namics.  (3)  Bio-sensing  is  based  on  molecular 
principles  of  recognition  to  identify  objects, 
environmental  conditions,  and  progression  in 
a  process.  (4)  Bio-dynamics  implements 
mathematical  models  of  cell  growth  and  fluid- 
dynamic  properties  of  biological  solutions. 
The  principles  are  implemented  in  a  simple 
world  model  of  the  human  vascular  system 
and  a  biomedical  problem  that  involves  an 
infection  by  Neisseria  meningitides  where  the 
biological  characters  are  white  and  red  blood 
cells  and  Neisseria  ceils.  Our  case  studies 
show  that  the  system  can  be  used  for  public 
health  education,  biomedical  simulation  and 
creative  problem  solving. 
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VIRONMENTS  FOR  THE  TREATMENT 
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Daniela  Villani,  Ph.D.  1  Enrico  Molinari, 

Psy.D.  Giuseppe  Rliva,  Ph.D.^'^ 
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cal  Sciences  LITA  Viaiba,  University  of  Milan,  Mi¬ 
lan,  Italy 

“’Clinical  Psychology  Lab,  Istituto  Auxologico  Itali¬ 
ano,  Verbania,  Italy 

The  cognitive  behavioral  therapy  (CBT)  is  still 
considered  the  best  approach  in  the  treatment 
of  eating  disorders,  but  it  could  present  differ¬ 


ent  limitations  related  to  costs  of  typical  be¬ 
havioral  procedures  or  difficulty  with  cognitive 
techniques.  Virtual  Reality  has  already 
shown  its  possible  utility  in  enhancing  the 
CBT  for  different  mental  problems.  Rationale 
and  protocols  about  the  use  of  Virtual  Real¬ 
ity-enhanced  treatments,  named  Integrated 
Experiential  Therapy  (lET),  will  be  explained. 
The  major  aim  of  this  presentation  is  the  de¬ 
scription  of  a  new  paradigm  of  virtual  envi¬ 
ronments  (VEs):  new  possibilities  for  different 
clinical  protocols  will  be  discussed.  The  real 
novelty  of  this  new  generation  of  VEs  is  the 
high  level  of  flexibility  and  plasticity  allowing 
therapists  to  save  different  versions  accord¬ 
ing  to  the  specific  users’  needs  and  features. 
Moreover  it  is  possible  to  carry  on  systemic, 
dynamic  or  cognitive-behavioural  approaches 
using  different  aspects  and  nuances  of  VEs. 

This  research  is  part  of  the  Prof.  Giuseppe 
Riva's  NEUROTIV  project  -  Immersive  virtual 
telepresence  managed  care  for  the  assess¬ 
ment  and  rehabilitation  in  neuro-psychology 
and  clinical  psychology. 
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The  main  goal  of  this  work  is  the  description 
of  a  new  Virtual  Reality-enhanced  treatment 
named  Integrated  Experiential  Therapy  (lET) 
for  the  treatment  of  Obesity  and  Binge  Eating 
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Disorder.  lET  for  eating  disorders  and  obesity 
is  a  relatively  short-term,  integrated,  patient 
oriented  approach  that  focuses  on  individual 
discovery.  Integrated  Experiential  Therapy  is 
an  integrated  approach  ranging  from  cogni¬ 
tive-behavioural  therapy  to  virtual  reality  ses¬ 
sions  for  the  treatment  of  eating  disorders  and 
obesity.  In  this  approach  VR  is  mainly  used  to 
modify  body  image  perceptions.  The  use  of 
VR  offers  two  key  advantages.  On  one  side,  it 
is  possible  to  integrate  all  different  methods 
(cognitive,  behavioural,  and  experiential)  com¬ 
monly  used  in  the  treatment  of  body  experi¬ 
ence  disturbances  within  a  single  virtual  ex¬ 
perience.  On  the  other  side,  VR  can  be  used 
to  induce  in  the  patient  a  controlled  sensory 
rearrangement  that  unconsciously  modifies 
his/her  bodily  awareness  (body  schema).The 
treatment  is  a  4-6-week  inpatient  treatment 
and  it  is  administered  by  therapists  having  a 
cognitive-behavioural  orientation  who  work  in 
conjunction  with  a  psychiatrist  as  far  as  the 
pharmacological  component  is  concerned. 
The  individual  work  regards  assessment  by 
means  of  psychometric  tests,  weekly  suppor¬ 
tive  psychological  talks,  7  sessions  for  as¬ 
sessment,  and  therapy  carried  out  using  Vir¬ 
tual  Reality  (VR),  and  psycho- 
pharmacological  assessment  and  control.  The 
psychological  group  therapy  is  based  on 
weekly  group  meetings  (closed  group  of  5/6 
persons)  of  two  hours  each.  The  work  group 
aims  both  at  training  for  development  and  ac¬ 
quisition  of  assertive  skills,  and  at  training  for 
assessment  and  consolidation  of  motivation. 
Moreover,  the  subjects  participate  to  both  bi¬ 
weekly  psycho-nutritional  groups  held  by  nu¬ 
tritionists  and  to  daily  group  sessions  of  physi¬ 
cal  activity.  In  order  to  verify  the  efficacy  of 
lET  we  are  realizing  a  clinical  trial  with  a  sam¬ 
ple  of  obese  patients  (with  and  without  Binge 
Eating  Disorder).  Subjects  were  randomly 
assigned  to  the  experimental  group  and  to  the 
three  control  groups.  Subjects  were  assessed 
by  one  of  three  independent  assessment  clini¬ 
cians  who  were  not  involved  in  the  direct  clini¬ 
cal  care  of  any  subject.  All  subjects  were  as¬ 
sessed  at  pre-treatment  and  upon  completion 
of  the  clinical  trial.  Patients  were  administered 
a  battery  of  outcome  measures  assessing 
eating  disorders  symptoms,  attitudes  toward 
food,  body  dissatisfaction,  level  of  anxiety, 
motivation  for  change,  level  of  assertiveness 
and  general  psychiatric  symptoms. 

In  summary,  the  preliminary  results  show  that 
the  virtual  simulation  of  demanding  real  life 
situations  is  useful  to  improve  patient’s  aware¬ 


ness,  body  satisfaction,  eating  control,  social 
skills,  self-esteem,  and  motivation  to  change. 
In  particular,  pre-treatment/post-treatment 
comparison  seems  to  indicate  that  Integrated 
Experiential  Therapy  is  more  effective  than 
traditional  approaches  in  the  treatment  of 
Obesity  and  Binge  Eating  Disorder.  Com¬ 
plete  results  will  be  presented  in  the  confer¬ 
ence. 

This  research  is  part  of  the  Prof.  Giuseppe 
Riva's  NEUROTIV  project  -  Immersive  virtual 
telepresence  managed  care  for  the  assess¬ 
ment  and  rehabilitation  in  neuro-psychology 
and  clinical  psychology. 
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COGNITIVE  MECHANISMS  UNDERLY¬ 
ING  VIRTUAL  REALITY  EXPOSURE’S 
EFFICACY  IN  THE  TREATMENT  OF 
ARACHNOPHOBIA 
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'  Ottawa  University,  Ottawa,  Ontario,  Canada 
^  University  of  Quebec  in  Outaouais,  Gatineau, 
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Research  status:  Completed. 

Virtual  exposure  therapy  for  phobias  is  a 
growing  field  of  interest.  Studies  have  begun 
to  demonstrate  its  efficacy,  which  is  at  least 
equivalent  to  in  vivo  exposure.  However, 
though  a  majority  of  studies  have  addressed 
efficacy  itself,  few  have  attempted  to  under¬ 
stand  the  treatment  mechanisms  underlying 
this  efficacy.  In  the  case  of  traditional  ther¬ 
apy,  two  models  are  still  the  object  of  a 
strong  debate:  the  information  processing 
model  and  the  perceived  self-efficacy  model. 
Interestingly,  this  debate  is  still  fuelled  by 
their  authors’  attempts  to  prove  the  predictive 
superiority  of  their  model  by  using  predicted 
variables  that  are  in  fact  different.  Actually, 
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no  study  has  yet  directly  compared  the  most 
likely  predictors  of  change,  for  virtual  expo¬ 
sure. 

The  goal  of  this  study  was  to  contrast  the  pre¬ 
dictive  value  of  different  treatment  mecha¬ 
nisms  for  specific  phobias.  The  hypotheses 
are  the  following.  Changes  in  information 
processing  (measured  with  a  pictorial  Stroop 
task)  will  better  predict  changes  in  anxiety 
during  a  behavioral  avoidance  test,  while 
changes  in  perceived  self-efficacy  will  better 
predict  changes  in  avoidance  behaviors  dur¬ 
ing  a  behavioral  avoidance  test. 

Twenty-eight  adult  arachnophobics  were  as¬ 
sessed  for  standardized  inclusion  and  exclu¬ 
sion  criteria.  General  outcome  and  specific 
processes  measures  included  questionnaires 
and  a  pictorial  Stroop  task.  The  main  out¬ 
come  and  predicted  variables  were  based  on 
a  behavioral  avoidance  test  (BAT).  Avoidance 
behavior  was  measured  by  participants’  ca¬ 
pacity  to  approach  a  live  tarantula  during  the 
BAT  and  anxiety  was  measured  by  recording 
their  heart  rate  during  the  first  minute  of  the 
task.  All  measures  were  completed  before 
treatment  (session  1)  and  at  post-treatment 
(session  7).  After  explanations  about  the  cog¬ 
nitive-behavioral  rationale  for  phobias  and  an 
initiation  to  the  virtual  reality  equipment 
(session  2),  participants  went  through  virtual 
exposure  for  five  sessions  (60  minutes  each) 
in  various  virtual  environments  with  spiders. 

Repeated  measures  ANOVAs  on  outcome 
measures  revealed  that  therapy  had  a  posi¬ 
tive  impact.  Analyses  made  on  the  pictorial 
Stroop  task  showed  that  information  process¬ 
ing  of  spider-related  stimuli  is  faster  after 
treatment,  which  also  indicates  therapeutic 
success.  Psychophysiological  data  also 
showed  a  positive  change  after  treatment, 
suggesting  a  decrease  in  anxiety.  In  them¬ 
selves,  these  results  represent  new  contribu¬ 
tions  to  the  field.  Hierarchic  regressions  were 
about  to  be  made  as  this  abstract  was  submit¬ 
ted:  results  will  be  available  in  the  presenta¬ 
tion.  These  regressions  will  be  used  to  assess 
the  respective  predictive  capacities  of  each 
key  variable  for  therapeutic  change. 

This  innovative  study  brings  fascinating  infor¬ 
mation  about  the  various  and  deep  impacts  of 
VRE  at  physiological,  information  processing 
and  cognitive  levels.  Therefore,  VRE  does 
provoke  significant  clinical  and  statistical 
therapeutic  change  for  people  suffering  from 


arachnophobia.  It  also  brings  elements  of  a 
better  understanding  of  the  distinct  contribu¬ 
tion  of  each  predictive  variable  in  the  psycho- 
pathological  models  for  phobias.  Interest¬ 
ingly,  this  information  can  be  applied  to  both 
in  vivo  and  virtual  reality  exposure.  Likewise, 
both  clinicians  and  researchers  can  benefit 
from  that  knowledge,  in  order  to  better 
choose  the  exposure  treatment  targets  or 
procedures,  according  to  the  type  of  changes 
they  want  to  observe  in  clients. 
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on  adolescent  mental  health  care.  Details  are 
given  of  preliminary  clinical  trials  of  a  custom 
developed  game  and  of  ongoing  research 
directions. 

Reports  show  that,  although  mental  health 
problems  increase  during  adolescence,  the 
majority  of  disturbed  adolescents  do  not  re¬ 
ceive  professional  mental  health  care  and  of 
those  who  do  fewer  still  fully  engage  with  the 
therapeutic  process  (Offer  et  al.  1991;  US 
Surgeon  General  1999).  Adolescents  are  gen¬ 
erally  more  private  and  self-conscious  than 
either  younger  children  or  adults.  Typically 
therapy  is  imposed  upon  them  (usually  by 
their  parents)  and  because  of  this  they  are 
less  willing  to  accept  it.  Play  therapists  often 
engage  younger  children  using  materials  such 
as  puppets,  storybooks,  and  construction  ma¬ 
terials.  Adolescents  can  be  resistant  to  these 
methods,  not  liking  to  be  treated  as  children. 
Equally,  many  react  confrontationally  or  not  at 
all  to  direct  dialogue  (Sharry  2004).  However 
adolescents  often  show  a  great  interest  in 
computer  games.  A  recent  UK  survey  re¬ 
ported  that  53%  of  eleven  to  fourteen  year 
olds  play  games  four  times  a  week  or  more 
(McFarlane  et  al.  2002).  Computer  games 
currently  constitute  a  client-centred  approach 
to  adolescent  psychotherapy. 

Personal  Investigator  (PI)  is  a  3D  computer 
game  specifically  designed  to  help  adoles¬ 
cents  with  mental  health  problems.  It  incorpo¬ 
rates  a  goal-oriented,  strengths  based  model 
of  psychotherapy  called  Solution  Focused 
Therapy  (SFT).  Adolescents  play  the  role  of  a 
personal  investigator  hunting  for  the  clues  that 
will  help  them  solve  personal  problems.  By 
engaging  adolescents,  in  a  client-centred 
way,  it  aims  to  build  stronger  therapeutic  rela¬ 
tionships  between  therapists  and  adolescents. 
PI  is  the  first  game  to  integrate  this  estal- 
ished  psychotherapy  approach  into  an  engag¬ 
ing  3D  game.  PI  differs  from  previous  thera¬ 
peutic  games  by  encouraging  the  adolescent 
to  create  a  written  record  of  their  own  discov¬ 
eries.  The  player  creates  a  virtual  detective 
notebook  in  which  they  write  down  all  their 
goals,  objectives,  ideas,  and  thoughts.  Upon 
completing  the  game,  they  receive  a  printout 
of  their  notebook.  Clark  et  al.  1984  reported 
benefits  of  having  a  tangible  output  from  a 
game.  A  pilot  study  of  the  game  has  been 
conducted  with  four  adolescents,  referred  to 
clinics  for  issues  including  anxiety  and  behav¬ 
iour  problems,  attempted  suicide,  and  social 
skills  difficulties.  Initial  results  indicate  that 


playing  PI  in  sessions  is  very  helpful  in  en¬ 
gaging  adolescents.  It  can  increase  the 
amount  of  dialogue  between  therapists  and 
adolescents  and  help  in  setting  therapeutic 
goals.  The  use  of  3D  had  an  empowering 
effect,  allowing  the  adolescent  more  control 
over  the  pacing  and  direction  of  the  thera¬ 
peutic  process.  For  full  details  of  the  theoreti¬ 
cal  foundations,  design,  and  pilot  study  of  PI 
see  Coyle  et  al.  2005. 

PI  implemented  a  therapeutic  model  in  an 
open  manner,  not  tailored  to  address  specific 
mental  health  issues  or  specific  adolescent 
cases.  One  ongoing  aim  of  the  Virtually 
Healthy  project  is  to  create  a  toolkit  that  al¬ 
lows  non-programming,  professional  thera¬ 
pists  to  adapt  and  create  issue  specific  com¬ 
puter  games.  The  system  should  be  usable 
by  therapists  with  wide-ranging  levels  of 
computer  literacy  and  be  adaptable  on  many 
levels,  from  small  fine-tuning  of  existing 
games  to  the  creation  of  new  games. 
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The  first  purpose  of  our  investigation  is  to  cre¬ 
ate  a  screening  protocol  where  repeatable 
and  parametric  facial  stimuli  are  presented 
interactively  to  patients  in  order  to  character¬ 
ize  and  later  identify  their  respective  mental 
disorders  using  their  measured  responses. 

We  would  like  to  prove  that  depressed  pa¬ 
tients  may  be  differentiated  based  on  the  deg¬ 
radation  in  performance  of  recognition. 

We  have  generated  24  faces  (12  male  -  12 
female)  expressing  the  6  basic  emotions  in 
two  different  ways  (shown  below),  and  2  neu¬ 
tral  faces  (1  male  -  1  female).  We  have 
shown  these  faces  to  117  people.  We  have 
accepted  a  picture  if  the  recognition  rate  was 
higher  than  70%.  Finally  we  have  chosen  the 
best  recognized  picture  of  each  emotion  and 
used  these  7  pictures  in  the  experiment.  In 
our  experiments  we  evaluated  26  depressed 
patients  and  used  frontal  views  of  the  two  3D 
animated  faces  and  asked  the  subjects  to 
identify  the  emotion.  We  generated  the  20-40- 
60-80-100%  (expression  level  of  the  emo¬ 
tions)  sequence  of  each  validated  face 
(5x7=35  pictures). 

ANOVA  evaluation  of  the  data  shows  that  de¬ 
pressed  patients  consistently  did  not  show 
differentiable  degradation  in  overall  perform¬ 
ance  when  compared  to  the  control  group. 
Furthermore  we  found  that  the  higher  the 
level  of  education  the  more  significantly 
higher  the  recognition  rate.  We  have  found 
significant  difference  between  depressed  and 


control  group  points  in  the  recognition  of  sad¬ 
ness  at  100%  of  expression  and  in  the  recog¬ 
nition  of  happiness  at  60%  of  expression.  In 
each  case  depressed  patients  reached 
higher  rates. 
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CONSULTATION  THROUGH  VIDEO- 
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Telepsychiatry  can  be  conceived  as  an  inte¬ 
grated  system  of  mental  health  care  delivery 
that  employs  telecommunications  and  com¬ 
puterized  information  technology  as  an  alter¬ 
native  to  face-to-face  conventional  alterna¬ 
tive.  Videoconferencing  is  the  central  tech¬ 
nology  currently  used  in  telepsychiatry,  since 
it  permits  live,  two-way  interactive,  full-colour, 
video,  audio,  and  data  communication.  The 
effectiveness  of  a  telepsychiatry  service 
through  videoconferencing  was  evaluated 
from  a  users’  perspective.  Ninety  (30  male, 
60  female)  subjects  completed  the  Symptom 
Rating  Checklist-90-Revised  (SCL-90-R)  on 
three  occasions,  once  before  the  first  tele¬ 
consultation  and  twice  during  telepsychiatry 
treatment.  Patients  and  psychiatrist  com¬ 
pleted  the  Clinical  Global  Impression  Scale- 
Severity  Index  (CGI)  after  each  teleconsulta¬ 
tion.  Twenty  five  percent  of  patients  were 
aged  less  than  30  years,  44%  were  aged 
between  30  and  45  years,  24%  between  45 
and  60  years,  while  7%  were  aged  over  60 
years.  Anxiety  disorders  were  the  more 
prevalent  diagnosis  (43.8%),  followed  by  de¬ 
pression  (36%),  schizophrenia  (7.9%),  sub¬ 
stance  disorders  (6.1%),  and  personality  dis¬ 
orders  (5.6%).  The  patients'  mean  SCL-90-R 
scores  decreased  significantly  over  time,  in¬ 
dicating  less  psychiatric  distress.  Both  pa¬ 
tients  and  psychiatrist  reported  patient  clini¬ 
cal  significant  improvement  over  time  as  as¬ 
sessed  by  the  CGI.  The  mean  number  of 
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teleconsultations  in  patients  discharged  was 
6.  Telepsychiatry  showed  to  be  an  effective 
mean  of  delivering  mental  health  services  to 
populations  living  in  remote  areas  with  limited 
resources. 
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A  telepsychiatry  service,  using  a  videoconfer¬ 
encing  system,  was  established  to  provide 
psychiatric  consultations  for  the  population  of 
La  Gomera,  in  the  Canary  Islands.  During  the 
first  year  of  routine  operation,  a  total  of  90 
patients  had  90  initial  and  224  follow-up  tele¬ 
consultations.  The  main  reason  for  the  con¬ 
sultation  identified  in  the  general  practitioner's 
referral  form  was  to  establish  a  diagnosis 
(70%  of  patients);  the  second  most  common 
reason  was  the  management  of  a  previously 
diagnosed  patient  (20%  of  patients).  Accord¬ 
ing  to  the  results  of  a  questionnaire  survey, 
patients'  acceptance  of  and  satisfaction  with 
the  technology  was  very  high.  In  their  first 
teleconsultation,  about  a  third  of  them  said 
that  they  experienced  some  initial  inconven¬ 
ience,  but  this  disappeared  after  a  few  min¬ 
utes.  During  the  teleconsultations,  90%  of 
patients  felt  understood  and  could  explain 
all  their  worries  and  fears.  Ninety  percent  of 
the  patients  considered  that  they  received  the 
follow-up  care  they  required  and  felt  satisfied 
with  the  attention  received.  Only  nine  out  of 
ninety  patients  (10%)  stated  they  liked  the 
'real'  doctor  better  than  the  'television  doctor'; 
forty  five  (50%)  said  they  had  no  preference, 
and  thirty  five  (40%)  expressed  their  prefer¬ 
ence  to  telepsychiatry. 
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Research  status:  This  abstract  presents  the 
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to  date,  21  subjects. 

Background/Problem 

Posttraumatic  Stress  Disorder  (PTSD)  is  a 
prevalent  and  often  persistent  mental  illness 
which  affects  approximately  7.8%  of  the  U.S. 
population.^  in  the  aftermath  of  the  World 
Trade  Center  attacks,  studies  of  residents  of 
New  York  City  have  found  increased  rates  of 
PTSD,^  with  rates  among  disaster  workers 
possibly  higher.  Expert  treatment  guidelines 
for  PTSD  recommend  cognitive-behavioral 
treatment  with  exposure  therapy.^  However, 
due  to  avoidance  symptoms  inherent  in 
PTSD,  some  patients  refuse  to  or  are  unable 
to  engage  their  emotions  in  treatment, 
thereby  limiting  its  efficacy.  The  use  of  vir¬ 
tual  reality  (VR)  provides  a  greater  possibility 
of  generating  patient  involvement  through  the 
multiplicity  of  sensory  cues  that  it  af¬ 
fords.  The  goal  of  this  study  is  to  evaluate 
the  efficacy  of  the  use  of  VR  in  the  treatment 
of  PTSD  directly  resulting  from  the  World 
Trade  Center  attacks  on  September  11, 
2001. 

Method/Tools 

Enrolled  subjects  meet  diagnostic  criteria  for 
PTSD  and  directly  witnessed  at  least  part  of 
the  attacks  of  September  ll*'^.  Subjects  in 
the  treatment  group  receive  treatment  based 
upon  a  14  -week  protocol  that  incorporates 
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exposure  using  virtual  reality  with  other  cogni¬ 
tive-behavioral  techniques.  The  subject  views 
scenes  of  the  World  Trade  Center  attacks 
through  a  3-D  headset  that  progress  in  sever¬ 
ity  culminating  in  two  planes  hitting  the  towers 
and  their  subsequent  collapses,  with  accom¬ 
panying  sounds  of  screaming  and  si¬ 
rens.  Subjects  complete  self-report  question¬ 
naires  weekly  and  are  assessed  prior  to  and 
following  the  treatment  by  an  experienced 
clinician.  Subjects  in  the  waitlist  control  are 
evaluated  at  baseline  and  again  after  14 
weeks,  at  which  time  they  are  offered  treat¬ 
ment. 

Results 

To  date,  7  subjects  have  completed  the  treat¬ 
ment  protocol  and  14  have  been  enrolled  in 
the  waitlist  condition.  This  sample  was  91% 
male  with  a  mean  age  of  43  (SD=9).  Notably, 
in  the  treatment  group  5  of  the  7  subjects 
were  disaster  workers  and  6  of  the  7  subjects 
had  failed  to  respond  to  imaginal  exposure 
therapy  (i.e.,  had  been  in  prior  treatment  for 
their  PTSD  symptoms  but  still  suffered  signifi¬ 
cantly  from  PTSD  with  a  mean  baseline 
CAPS  score  of  78  +/-  24).  Data  analysis 
shows  that  scores  on  the  CAPS  decreased  by 
an  average  of  28  points  for  individuals  in  the 
VR  group  and  decreased  by  an  average  of  5 
points  for  individuals  in  the  waitlist  control 
(P<-05). 

Conclusion 

This  preliminary  data  suggests  that  virtual 
reality  is  an  effective  treatment  for  PTSD. 
Novelty/Discussion:  This  preliminary  study  is 
the  first  to  show  that  virtual  reality  is  an  effica¬ 
cious  treatment  for  individuals  suffering  from 
PTSD  following  terrorism  and,  to  our  knowl¬ 
edge,  is  only  the  second  study  of  PTSD  using 
virtual  reality.  Treatment  failures  using  imagi¬ 
nal  exposure  may  be  partially  due  to  the  in¬ 
ability  of  the  patient  to  emotionally  engage  in 
the  exposure  work.  Virtual  reality  appears  to 
address  this  problem  by  enhancing  the  pa¬ 
tient's  capacities  with  visual,  auditory,  and 
even  haptic  computer-generated  experiences, 
thereby  facilitating  the  patient's  emotional  en¬ 
gagement  in  the  exposure  treatment. 
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Interacting  in  an  immersive  virtual  environ¬ 
ment  we  refer  to  the  real  world.  The  user  gets 
to  expect  real  world  behaviour  from  virtual 
objects  and  functions.  We  explore  the  idea  of 
bringing  aspects  of  product  design  to  the  de¬ 
velopment  of  input  devices  and  interaction 
techniques  for  virtual  environments.  We  use 
ergonomics  and  product  language  to  design 
the  physical  input  device,  the  virtual  represen¬ 
tation  and  the  connection  between  real  and 
virtual  parts.  This  connection,  which  we  cail 
amalgamation,  is  the  most  important  element 
of  the  design,  since  it  defines,  if  the  user 
rather  interacts  with  virtual  objects  as  “virtual 
products”  (via  an  input  device),  or  rather  inter¬ 
acts  with  an  input  device  as  a  “real  product” 
controlling  virtual  content. 
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POTENTIAL  MECHANISMS  UNDERLY¬ 
ING  THE  EFFICACY  OF  VR  TO  RE¬ 
DUCE  ACUTE  PAIN:  A  LITERATURE 
REVIEW 

Stephanie  Dumoulin,  B.Sc.,  Stephane  Bou¬ 
chard,  Ph.D.  &  Vicky  Rivard,  Ph.D. 

Cyberpsychology  Lab  of  the  University  of  Quebec 
in  Outaouais 

Research  status;  Completed. 

Pain  is  now  considered  a  complex  subjective 
phenomenon  that  involves  sensorial,  motiva¬ 
tional,  cognitive,  and  emotional  dimensions. 
Recent  studies  have  now  shown  that  virtual 
reality  (VR)  can  be  used  to  control  and  reduce 
acute  pain;  probably  because  of  it's  potential 
for  distracting  attention  away  from  the  pain. 
But  many  factors  may  influence  the  efficacy  of 
VR  to  help  manage  pain.  The  aim  of  this  lit¬ 
erature  review  is  to  summarize  the  studies 
that  assessed  the  efficacy  of  VR  in  experi¬ 
mental  and  clinical  pain  management  in  order 
to  highlight  psychological  variables  that  may 
be  involved  in  the  mechanisms  of  pain  man¬ 
agement  using  VR. 

Method 

This  systematic  review  followed  a  standard 
and  structured  research  approach.  It  has  the 
advantages  of  being  rigorous,  replicable,  and 
the  results  can  easily  be  presented  in  a  poster 
format.  Journal  databases  such  as  MedLine, 
PsycINFO  and  Web  of  Science  were 
searched  with  key  words  such  as  «pain»  or 
«pain  management))  were  crossed  with  key 
words  such  as  «virtual))  or  «virtual  reality)). 
More  than  110  articles  were  found.  Most  of 
them  were  theoretical.  But  17  empirical  stud¬ 
ies  using  VR  to  manage  pain  were  found.  Six 
used  rigorous  experimental  protocols  and  11 
were  more  exploratory  and  had  no  experi¬ 
mental  protocol. 

Results  and  Conclusion 

Most  studies  showed  an  important  and  statis¬ 
tically  significant  reduction  in  pain.  Although 
distraction  is  considered  the  key  ingredient  to 
explain  theses  results,  a  detailed  analysis  re¬ 
veals  that  several  factors  might  be  involved. 
These  factors  could  be  grouped  in  three  cate¬ 
gories:  task  relevant  (e.g.,  attention  required, 
task  complexity,  and  emotional  content),  indi¬ 


vidual  (e.g.,  sense  of  presence,  hypnotisabil- 
ity,  self-efficacy,  and  outcome  expectations) 
and  pain  factors  (e.g.,  pain  intensity).  We 
believe  that  by  highlighting  explicitly  in  this 
poster  the  variables  that  may  lead  to  pain 
management,  VR  researchers  can  narrow 
more  effectively  their  search  for  treatment 
moderators  and  mediators. 
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Natalia  Fernandez,  PhD  candidate 

Universidad  Politecnica  de  Madrid 

One  of  the  main  challenges  that  developed 
societies  face  is  the  prevention  of  non- 
communicable  diseases  such  as  diabetes  or 
obesity.  These  diseases  are  responsible  for 
60%  of  the  registered  global  deaths  and  for 
45%  of  the  global  burden  of  disease.  Un¬ 
healthy  diet,  together  with  physical  inactivity 
and  smoking,  are  among  the  key  risk  factors 
for  the  development  of  these  illnesses.  The 
fact  that  these  are  preventable  leads  directly 
to  the  elaboration  of  prevention  strategies 
that  aim  to  empower  and  motivate  the  citi¬ 
zens  to  be  responsible  for  their  own  health 
by,  among  others,  providing  them  with  quality 
and  personalised  information.  As  the  purpose 
is  to  incorporate  healthy  lifestyles  to  the  us¬ 
ers’  daily  activities,  motivational  aspects 
need  to  be  seriously  taken  into  account. 
Hence,  the  way  the  information  is  presented 
to  the  user  must  be  carefully  analysed. 

The  development  of  a  virtual  reality  tool  helps 
to  motivate  the  user  and  also  improves  the  e- 
learning  process.  The  work  presented  in  this 
paper  contributes  partially  to  solve  this  situa¬ 
tion  by  offering  the  users  personalised  infor¬ 
mation  for  their  own  self-care  and  further¬ 
more,  by  motivating  them  to  make  use  of  this 
information  and  take  control  of  their  decisions 
regarding  their  lifestyle  through  the  use  of  a 
virtual  reality  tool. 

The  tool,  based  in  X3D  (extensible  3D),  a 
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virtual  3D  technology  for  the  Internet,  is  inte¬ 
grated  in  an  e-learning  environment  which  is 
also  interactive  and  immersive.  A  training  ac¬ 
tivity  in  the  system  consists  of  an  Internet  mul¬ 
timedia  session  that  shows  personalised  infor¬ 
mation  by  means  of  an  attractive  visuai  inter¬ 
face.  It  is  worth  mentioning  that  the  tool  not 
only  provides  citizens  specific  information 
adapted  to  their  profiie  and  needs  but  also  the 
interface  is  personaiised  accordingly.  The  in¬ 
formation  managed  throughout  the  system, 
including  the  virtuai  scenarios,  has  been 
stored  using  the  XML  format. 

Moreover,  in  order  to  perform  the  personalisa¬ 
tion  of  the  tool  for  every  single  user,  the  sys¬ 
tem  has  been  programmed  to  be  "intelligent." 
The  process  is  started  by  obtaining  the  user 
profile.  This  is  created  the  first  time  the  citizen 
accesses  the  tool  by  filling  in  a  structured 
questionnaire  about  nutritional,  statistical,  mo¬ 
tivational  and  physical  activity  data.  By  proc¬ 
essing  this  data,  the  information  that  suits  the 
user  needs  and  preferences  best,  is  selected 
from  a  native  XML  database.  In  addition,  the 
information  is  presented  to  the  users  in  a  dif¬ 
ferent  way  depending  on  their  motivational 
state,  which  is  revealed  by  applying  the 
"Stages  of  Change"  methodology. 

In  order  to  measure  the  user  response  to  the 
system,  a  survey  is  compiled  after  the  first 
training  session.  The  evaluation  of  the  results 
allows  us  to  conclude  that  Virtual  Reality  is 
perceived  as  an  interesting  and  easy  to  use 
tool.  Besides,  the  users  state  that  the  relation 
between  the  scenarios  developed  and  the 
information  provided  in  each  of  them  was  tai¬ 
lored  to  their  needs  and  the  contents  were 
marked  as  a  good  educational  tool  to  adopt  a 
healthy  lifestyle. 
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Jose  Gutierrez-Maldonado,  PhD  ,  Marta 
Ferrer-Garcia,  PhD  candidate 

University  of  Barcelona,  Spain 

This  paper  describes  the  effectiveness  of  vir¬ 
tual  environments  to  elicit  emotional  re¬ 
sponses  in  eating  disordered  patients.  This 
study  is  part  of  a  wider  research  project  ana¬ 
lysing  the  influence  of  the  situation  to  which 
subjects  are  exposed  on  their  performance  on 
body  image  estimation  tasks. 

Though  it  seems  self-evident  that  there  is  a 
close  relation  between  eating  disorders  (ED) 
and  aitered  body  image,  in  the  literature  on 
the  subject  no  clear  association  has  in  fact 
been  established,  and  indeed  the  results  of 
the  studies^' are  often  contradictory  or 
inconclusive.  A  number  of  hypotheses  have 
been  proposed  to  account  for  these  results. 
Some  authors  have  stressed  the  fact  that 
body  image  may  be  to  a  certain  extent  more  a 
state  than  a  traiP  ®  and  may  change  accord¬ 
ing  to  situational  or  emotional  variables.  Sev¬ 
eral  studies  have  anaiysed  the  possible  im¬ 
pact  of  exposure  to  specific  objects  or  situa¬ 
tions  on  the  stability  of  the  body  image.®'^  ®  ®' 
10,11 

In  this  study  we  designed  several  virtual  envi¬ 
ronments  that  were  emotionally  significant  for 
subjects  with  ED  in  order  to  generate  differ¬ 
ent  levels  of  anxiety  and  variations  in  mood. 
Unlike  conventional  methods  (real  exposure 
to  the  situation,  exposure  to  photographs, 
exposure  via  guided  imagination,  and  so  on), 
virtual  reality  exposes  subjects  to  interactive 
three-dimensional  environments  that  simu¬ 
late  real  situation.  These  environments  have 
ecological  validity  but  also  permit  strict  con¬ 
trol  over  the  variables  and  the  recording  of 
data.  Virtual  reality  offers  many  of  the  advan¬ 
tages  of  the  conventional  methods  men¬ 
tioned  above,  and  aiso  overcomes  many  of 
their  drawbacks. 

Thirty  female  patients  with  eating  disorders 
were  exposed  to  six  virtual  environments:  a 
living-room  (neutral  situation),  a  kitchen  with 
high-calorie  food,  a  kitchen  with  low-calorie 
food,  a  restaurant  with  high-calorie  food,  a 
restaurant  with  low-calorie  food,  and  a  swim¬ 
ming-pool.  After  exposure  to  each  environ¬ 
ment  the  STAI-S  (a  measurement  of  state 
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pression)  were  administered  to  all  subjects. 

The  results  showed  significantly  higher  levels 
of  state  anxiety  in  the  kitchen  with  high-calorie 
food  (F=13.120:  p  =  0.001),  the  restaurant 
with  high-calorie  food  (F  =  14.954;  p  =  0.001) 
and  the  swimming-pool  (F  =  4.230;  p  =  0.049) 
than  in  the  neutral  environment.  Analysing 
the  scores  for  depression  obtained  on  the 
CDB,  significant  differences  again  appeared 
between  the  high-calorie  food  environments 
(F  =  7.187;  p  =  0.012  in  the  kitchen  and  F  = 
5.933;  p  =  0.021  in  the  restaurant)  and  the 
neutral  environment.  In  the  high-calorie  food 
situations  patients  with  ED  showed  a  more 
depressed  mood. 

Virtual  reality  thus  appears  to  be  a  valid  in¬ 
strument  particularly  useful  for  simulating  eve¬ 
ryday  situations  that  may  provoke  emotional 
reactions  such  as  anxiety  and  depression,  in 
patients  with  ED.  Virtual  environments  in 
which  subjects  are  obliged  to  ingest  high- 
calorie  food  provoke  the  highest  levels  of 
state  anxiety  and  depression.  Previous  stud¬ 
ies  have  shown  the  capacity  of  VR  to  elicit 
states  of  anxiety  in  patients  with  other  pa¬ 
thologies  too. 
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APPLICATIONS?  A  DISCUSSION  OF 
FIVE  STUDIES 

Raymond  A.  Folen,  Ph.D. 

Tripler  Army  Medical  Center,  Department  of 
Psychology 

Research  Status:  In  Progress 
Background/Problem 

Virtual  Reality  (VR)  interventions  for  behav¬ 
ioral  health  have  demonstrated  promise  for  a 
wide  variety  of  clinical  concerns.^  At  the 
same  time,  most  research  to  date  has  lacked 
control  conditions  and  comparisons  with  non- 
VR  environments  and  interventions,^  making  it 
more  challenging  to  justify  the  cost  of  estab¬ 
lishing  in-house  VR  services.  Research  being 
conducted  at  the  Pacific  Telehealth  &  Tech¬ 
nology  Hui  Virtual  Reality  Behavioral  Health 
laboratory  compares  VR  interventions  with 
active  treatment  controls  and  standard  of  care 
controls,  and  assesses  the  efficacy  of  differ¬ 
ent  display  formats.  Five  controlled  compari¬ 
son  study  protocols  designed  to  advance  the 
field  of  VR  in  behavioral  health  will  be  pre¬ 
sented  and  preliminary  findings  will  be  dis¬ 
cussed. 

Method 

Two  of  the  research  projects,  a  reactivity  to 
anger  stimuli  study  and  a  nicotine  cue  expo¬ 
sure  study,  evaluate  the  effectiveness  of  im¬ 
mersive  panoramic  video  virtual  environment 
(VE)  displays  in  evoking  physiological  and 
emotional  responses  to  anger  stimuli  or  smok¬ 
ing  cues  compared  to  flat  screen  video  dis¬ 
plays  of  the  same  content.  Outcome  meas¬ 
ures  include  standardized  questionnaires  and 
physiological  measures  of  heart  rate,  blood 
pressure,  skin  conductance,  temperature,  and 
respiration. 
The  third  study,  a  biofeedback  training  study, 
compares  the  effectiveness  of  three  biofeed¬ 
back  training  modalities  for  controlling  periph¬ 
eral  body  temperature.  Participants  are  ran¬ 
domly  assigned  to  one  of  three  training  condi¬ 
tions;  (1)  temperature  change  presented  on  a 
monitor  (standard  feedback  mode),  (2)  stan¬ 
dard  feedback  presented  through  a  HMD,  and 
(3)  immersive  VE  with  pictorial  feedback  dis¬ 
play  presented  through  a  HMD.  Outcome 
measures  include  the  maximum  temperature 
difference,  time  to  maximum  temperature, 
time  to  criterion  temperature,  duration  of  crite¬ 


rion  temperature,  and  rate  of  change. 
Chronic  pain  is  the  focus  of  the  fourth  study 
which  compares  the  effectiveness  of  three 
treatment  conditions  in  reducing  the  percep¬ 
tion  of  chronic  pain:  (1)  therapist  guided  im¬ 
agery  (standard  treatment),  (2)  audio-taped 
imagery,  and  (3)  therapist  guided  imagery 
enhanced  with  VR.  The  outcome  measures 
consist  of  pre-  and  post-  treatment  pain  per¬ 
ception,  self-efficacy  ratings  of  treatment  ef¬ 
fectiveness,  and  physiological  measures. 
The  fifth  study  compares  the  effectiveness  of 
VR  as  a  pain  and  anxiety  distraction  to  the 
standard  of  care  for  participants  undergoing 
a  cystoscopy  examination.  Outcome  meas¬ 
ures  include  the  degree  of  perceived  pain, 
pre-  and  post-procedure  anxiety,  and  physio- 
logical  measures. 
Results,  Conclusions,  and  Novelty:  The  five 
studies  will  be  discussed  in  terms  of  protocol 
design  and  anticipated  unique  contributions 
to  the  field  of  VR  in  behavioural  health.  Sev¬ 
eral  themes  will  be  highlighted,  including  (1) 
the  need  to  establish  the  incremental  benefit 
of  using  VR  versus  alternative  display  for¬ 
mats,  such  as  in  the  anger  reactivity,  nicotine 
and  biofeedback  studies,  (2)  the  need  for 
controlled  comparisons  as  exemplified  by  all 
of  the  studies,  (3)  the  need  to  generalize  find¬ 
ings  within  clinical  domains,  e.g.,  generaliz¬ 
ing  pain  analgesia  findings  for  cystoscopy 
examinations,  and  (4)  the  benefits  of  estab¬ 
lishing  experimental  results  that  may  be 
transportable  to  clinical  applications,  such  as 
the  use  of  VR-enhanced  thermal  biofeedback 
training  for  reducing  migraine  headaches  or 
anger  reactivity  study  protocols  for  anger 
management  programs.  Preliminary  stan¬ 
dardized  questionnaire  and  physiological 
findings  will  also  be  presented. 
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PHYSIOLOGIC  MONITORING  OF  SOL¬ 
DIERS:  PRESENT  AND  FUTURE 

Beau  J.  Freund,  Mark  J.  Buller,  William  A. 

Latzka  and  Karl  E.  FriedI 

U.S.  Army  Research  Institute  of  Environmental 
Medicine 

Wearable  medical  sensors  can  provide  real 
time  situational  awareness  to  the  soldier, 
commander,  and  medic.  Key  to  this  system  is 
the  algorithms  and  knowledge  displays  that 
link  and  interprets  sensor  information.  For  the 
first  time  ever,  the  U.S.  Army  has  a  prototype 
system  that  will  monitor  physiological  signals 
from  the  soldier  and  integrate  this  sensor  data 
through  algorithms  to  predict  health  status. 
This  Warfighter  Physiological  Status  Monitor- 
Initial  Capability  (WPSM-IC)  is  comprised  of  a 
medical  hub  that  hosts  a  personal  area  net¬ 
work  of  physiologic  and  medical  sensors 
(heart  rate,  respiration,  skin  and  core  tem¬ 
perature,  body  motion,  acoustic/impact,  sleep 
history,  fluid  intake)  and  algorithms.  The  algo¬ 
rithms  estimate  thermal,  hydration,  cognitive, 
life  signs,  and  wounding  status  from  the  sen¬ 
sors  distributed  around  a  soldier’s  body,  uni¬ 
form  and  equipment  and  from  other  models, 
databases,  and  contextual  information  (e.g., 
location,  weather,  mission,  individual  data). 
This  system  is  intended  as  one  component  of 
future  warrior  system  of  systems  with  differen¬ 
tial  displays  of  the  information  to  the  individ¬ 
ual,  the  unit  leader,  and  the  medic,  ranging 
from  aggregate  data  on  overall  status  mem¬ 
bers  of  the  unit  for  the  leader  to  specific  early 
triage  of  a  wounded  soldier  for  the  medic. 
This  physiological  monitoring  capability  lays 
the  groundwork  for  data  collection  for  further 
refinements,  as  well  as  testing  a  wide  range 
of  potential  useful  applications.  Further  re¬ 
search  with  this  system  will  expand  under¬ 
standing  of  limits  on  human  tolerance.  Appli¬ 
cations  may  include  safety  monitoring  in  haz¬ 
ardous  conditions,  training  to  learn  and  teach 
soldier  limits,  closed  loop  systems  for  control 
of  equipment  (e.g.,  microclimate  cooling,  exo¬ 
skeleton,  information  displays),  and  as  senti¬ 


nel  soldier  surveillance  systems.  This  sys¬ 
tem  also  provides  the  backbone  for  future 
neurocognitive  monitoring  enhancements. 
These  may  draw  from  currently  available 
sensors  such  as  voice  analysis,  pupil  re¬ 
sponses,  saccadic  eye  movements,  slow 
eyelid  closure,  electroencephalography,  non- 
invasive  biochemical  measures  (e.g.,  tear 
analytes),  and  nerve  conduction  velocity. 
The  challenge  will  be  the  research  studies  in 
realistic  and  stressful  conditions  that  lead  to 
development  of  valid  predictive  algorithms  of 
brain  function  and  behavior.  These  predic¬ 
tions  of  the  consequences  of  information 
overload,  psychological  trauma,  and  physio¬ 
logical  stress  consequences  to  brain  function 
will  be  central  to  future  adaptive  displays. 
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THE  VIRTUAL  REALITY  MIRROR: 
MENTAL  PRACTICE  WITH  AUG¬ 
MENTED  REALITY  FOR  POST¬ 
STROKE  REHABILITATION 
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russo'',  Giuseppe  RivaV 
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^Telecom  Italia  Learning  Services,  Rome,  Italy 

Research  status:  in  progress 

The  aim  of  the  project  is  to  apply  augmented- 
reality  technology  to  teach  motor  skills  to  pa¬ 
tients  suffering  from  stroke.  To  achieve  this 
goal  the  project  adopts  an  innovative  ap¬ 
proach  based  on  the  use  of  so-called  “motor 
imagery.”  Recent  studies  in  neuroscience 
have  provided  converging  evidence  that 
“imagining”  a  motor  action  involves  the  same 
brain  areas  involved  in  performing  the  ac- 
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tion.^  This  supports  the  idea  -  already  ex¬ 
ploited  by  sports  trainers  -  that  training  with 
motor  imagery  (mental  practice)  could  be  ef¬ 
fective  in  learning  new  motor  skills  or  in  pro¬ 
moting  motor  recovery  after  damage  to  the 
central  nervous  system.  Previous  clinical 
studies  have  shown  that  the  rehabilitation  of 
post-stroke  hemiplegic  patients  can  be  made 
more  effective  by  combining  physical  practice 
with  mental  practice.^’^  However,  for  many 
patients  who  suffer  from  damage  to  the  cen¬ 
tral  nervous  system,  mental  simulation  of 
movements  can  be  difficult,  even  when  the 
relevant  neural  circuitry  has  not  been  injured. 
Starting  from  these  premises,  we  have  de¬ 
signed  and  developed  an  augmented-reality 
workbench  (called  “VR  Mirror”)  to  help  post¬ 
stroke  hemiplegic  patients  evoking  motor  im¬ 
ages.  First,  the  movement  is  acquired  by  the 
system  from  the  healthy  arm.  Second,  the 
movement  is  being  mirrored  and  displayed  so 
that  the  patient  can  observe  and  see  as  if  the 
impaired  arm  is  performing  the  movement. 
Third,  the  patient  is  instructed  to  rehearse  in 
his/her  imagination  the  movement  he/she  has 
just  observed.  Last,  the  patient  has  to  perform 
the  movement  with  the  affected  arm.  The  sys¬ 
tem  is  currently  being  tested  to  see  if  it  does 
help  patients  to  recover  more  quickly  and  re¬ 
gain  control  of  arms  that  have  been  paralysed 
following  a  stroke.  Preliminar  clinical  out¬ 
comes  will  be  presented  during  the  confer¬ 
ence.  The  research  is  supported  by  the  EU 
project,  called  l-leaming  (Immersion/Imagery 
Enhanced  Learning'*),  which  is  funded  under 
the  FET  Program  (Future  and  Emerging 
Technologies®). 
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VIRTUAL  TECNOLOGIES  FOR  EX¬ 
TREME  ENVIROMENT  EFFECT  MITI¬ 
GATION 
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This  paper  examines  the  demands  of  a 
manned  system  operating  in  an  extreme  envi¬ 
ronment  where  there  is  the  need  to  enhance 
or  at  least  maintain  crew  health,  team  spirit 
and  cohesion  and  how  new  techniques  based 
on  Virtual  Reality  technologies  can  provide 
some  answers.  A  typical  example  is  a 
manned  mission  to  Mars  with  a  crew  of  4-8 
astronauts  where  it  is  mandatory  to  maintain 
their  psychological  health  for  a  mission  that  is 
envisaged  to  last  from  two  to  three  years. 
They  will  have  to  work  and  live  in  an  artificial 
environment  with  limited  resources  (for  exam¬ 
ple  communication  capability  with  ground,  due 
to  the  distances  involved,  cannot  be  a  direct 
two  way  system)  and  living  volumes.  The  pa¬ 
per  proposes  some  possible  fields  of  investi¬ 
gation.  It  will  also  consider  the  fact  of  how 
these  innovative  technologies  based  on  Vir¬ 
tual  Reality  will  have  to  be  developed  and 
tested. 
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Domotic  introduces  remarkable  advantages, 
above  all  to  impaired  persons,  and  elders,  but 
at  present  the  lack  of  an  actual  user-oriented 
design  occurs.  We  planned  and  put  in  act  an 
action-research,  now  in  progress,  in  the 
framework  of  the  "Domus"  project  (Tuscany 
Region  council  resolution  n.977/2003)  with 
the  participation,  in  the  first  phase,  of  10  cou¬ 
ples  affected  by  mobility  disorders,  each  one 
living  for  five  days  in  a  domotic  apartment. 

The  aims  are  to  set  up  a  methodology  of 
analysis  of  the  user-domotic  environments 
interaction  and  to  supply  results  concerning 
the  interaction  of  an  impaired  user  with  a  do¬ 
motic  environment,  with  the  aim  both  to  as¬ 
sess  the  system  and  to  propose  new  solu¬ 
tions. 

General  approach 

An  anthropological  and  ethno-methodological 
approach  was  followed,  in  terms  of  an  actual 
user-oriented  design.  Into  details,  the  domotic 
environments  design  must  follow  an  iterative 
procedure,  foreseeing  a  continuous  loop 
among  design,  ergonomic  evaluation,  and 
implementation.  This  ergonomic  stance  to 
evaluate  not  only  technical  and  functional  re¬ 
quirements  (conformance  testing)  but  also 
general  usability  and  usefulness  of  the  envi¬ 
ronment  (usability  evaluation),  considering 
both  the  physical  and  the  cognitive  affor- 
dance. 

Methodolgy 

The  evaluation  phases  foresee  the  analysis  of 
autonomy  project,  impairments  and  re¬ 
sources,  and  interaction  with  artefacts. 

Focusing  on  the  interaction  analysis  we  evalu¬ 
ated  usability,  efficacy,  efficiency,  satisfaction, 
usefulness,  and  pleasantness. 

Methods  and  techniques  were  constructive 
interaction  -  thinking  aloud  modality,  partici¬ 
pating  observation,  semi-structured  interview, 
video-recordings  of  all  the  interactions,  and 
analysis  of  the  reports  supplied  by  the  soft¬ 
ware  controlling  the  electronic  domotic  envi¬ 
ronments. 
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BEYOND  INTERFACES:  AN  INTE¬ 
GRATED  APPROACH  TO  THE  ERGO¬ 
NOMIC  ANALYSIS  OF  VIRTUAL  ENVI¬ 
RONMENTS  IN  PSYCHOTHERAPY  OF 
ANXIETY  DISORDERS 
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Research  Status;  In  Progress  (analysis  of 
first  clinical  trial  data) 

Background/Problem 

The  present  study  is  part  of  the  FIRB- 
NEUROTIV  project  whose  main  objective  is 
to  develop  -  on  the  basis  of  the  result  and 
applications  coming  out  from  VEPSY  Project, 
funded  by  EU  (Galimberti  et  al.  2004)  -  the 
technical  and  clinical  viability  of  using  Virtual 
Reality  Therapy  in  clinical  psychology  by 
means  of  portable  and  shared  VR  Systems. 
The  project  is  providing  enhancements  of 
Vepsy’s  tools  (Telemedicine  and  Portable 
tools)  for  the  treatment  of  patients,  new  clini¬ 
cal  trials  to  verify  their  viability,  action  pians 
for  dissemination  of  its  results  to  an  extended 
audience  and  an  integrated  approach  to  er¬ 
gonomic  analysis  of  3D  Virtual  Environments 
for  psychotherapy. 

Method/Tools 

To  face  the  aspects  connected  with  VR  envi¬ 
ronments’  usability  for  psychotherapeutic 
applications  mean  to  dare  a  double  chal¬ 
lenge  from  a  methodological  point  of  view: 
from  one  side,  the  need  to  adapt  and  to  inte- 
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grate  on  a  heuristic  basis  classic  usability 
evaluation  methods  to  specific  artifacts  such 
as  3D  Virtual  Environments  for  clinical  appli¬ 
cations  (Galimberti  and  Belloni,  2003);  from 
the  other  hand,  the  problems  arisen  by  inte¬ 
gration  of  expert  evaluation  of  VR  environ¬ 
ments  user-based  tests  carried  out  in  real 
context  of  use  (Riva,  2003;  Mantovani,  2000). 
To  face  these  challenges  we  have  chosen  to 
base  our  analytical  stance  upon  an  ethnome- 
todological  approach,  a  perspective  that  gives 
evidence  of  how  people,  in  specific  social 
situations,  are  able  to  solve  complex  tasks 
producing  shared  meanings  and  achieving 
their  goals  during  interaction.  According  to 
this  perspective,  the  methodological  objective 
consisted  also  in  the  identification  of  the  us¬ 
ability  requirements  of  the  specific  community 
of  practice  by  whom  Virtual  environments  are 
to  be  used  (Riva  and  Galimberti,  2001).  The 
virtual  environments  considered  were  the 
Panic  Disorders  and  Agoraphobia  VR  mod¬ 
ules  developed  in  the  framework  of  the  FIRB  - 
NEUROTIV  project. 

Data  had  been  produced  by  means  of  differ¬ 
ent  situations: 

•  functional  analysis  of  VR  environments 

•  observational  analysis  of  videotaped  pa- 
tient-VR  environments  interactions,  thera¬ 
pist-patient  interactions  and  therapist- 
patient-VR  environments  interactions  dur¬ 
ing  therapy  sessions 

•  Semi-structured  interviews  to  out-patients 

•  Usability  tests  performed  by  out-patients 

Different  data  analysis  technique  (Galimberti 
et  al.,  2004;  Krug,  2000),  both  quantitative 
and  qualitative,  are  presently  applied  to  data 
producted. 

Novelty/Discussion 

•  The  main  goal  of  the  study  is  to  go  ‘beyond 
interfaces,’  extending  ergonomic  analysis 
of  3D  virtual  environments  use  in  psycho¬ 
therapy  to  such  topic  as: 

•  their  pragmatic  context  of  use 

•  the  culture  of  use  featuring  the  community 
of  professional  users 

•  side-effects  and  by-products  of  their  appli¬ 
cation 

• 

Obviously  without  forgetting  the  outcomes  of 
traditional  usability  analysis.  The  research  is 
thus  intended  to  construct  -  and  not  of  simple 
application  -new  tools  for  an  ergonomic  analy¬ 
sis.  One  of  its  main  features  will  be  to  have  not 


only  a  descriptive  function,  but  to  find  out  solu¬ 
tions  to  make  the  VEs  systems  more  efficient. 
And  this  is  intended  to  improve  the  whole  in¬ 
teractive  process,  abandoning  both  artifact 
and  user-artifact  centered  interaction  in  favor 
of  a  ‘situated  and  context  sensible’  ergonomic 
analysis. 
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Research  Status:  Complete 

This  paper,  following  the  idea  of  Persuasive 
Technology  proposed  by  Fogg  (2003),  pre¬ 
sents  a  way  to  develop  a  Virtual  Reality  (VR) 
educational  system,  Playsafety,  capable  of 
having  an  impact  on  risky  behaviors  in  young 
people.  The  participative  design  of  Playsafety, 
its  application  in  the  field  and  its  first  evalua¬ 
tion  will  be  illustrated. 

Background 

Can  Virtual  Reality  work  as  a  means  to  pre¬ 
vent  drug  abuse  and  to  persuade  people  into 
adopting  safe  behaviors?  The  potentials  of 
VR  as  learning  or  health  care  environment 
have  long  been  investigated  (de  Jong,  1991; 
McComas  2002;  Riva,  2002).  Virtual  Environ¬ 
ments  offer  the  opportunity  to  explore  social 
situations  and  'try  out'  different  responses  to  a 
variety  of  events  (Kerr  et  al,  2002;  Alcaniz  M., 
2003).  As  simulations,  they  also  allow  to  cope 
with  novel  situations  (Gamberini  et  al.,  2003) 
and  explore  cause-and-effect  relationships  in 
a  safe,  non-threatening,  and  compelling  envi¬ 
ronment  (Fogg,  2003). 

Method  and  Tools 

A  first  step  in  the  creation  of  Playsafety  was 
an  ethnographic  investigation  of  dangerous 
night  situations.  A  six-month  participative  ob¬ 
servation  was  carried  out  in  the  clubs  of  a 
summer  holiday  location  in  Italy.  The  field 
notes  were  organized  as  ‘stories  about  people 
and  how  they  keep  on  their  projects’  (Carroll, 
2000). 

In  a  second  stage,  following  the  scenario- 
based  design  approach  (Carroll,2000),  four 
different  hazardous  situations  were  extracted 
from  the  notes  and  as  many  VEs  were  built 
with  3D  Studio  Max  4.2  and  Virtools  DEV  2.5. 
Environments  included  a  discotheque,  a  park, 
a  restroom  as  place  for  drug  and  alcohol 
abuse,  and  a  motorbike  ride  (safe  drive).  The 
heuristic  method  of  expert  evaluation 
(Nielson,  1993;  Mix  et  al.,1993)  was  deployed 
to  test  their  usability.  Within  each  environ¬ 
ment,  two  avatars  interacted  with  the  user. 
Participants  were  free  to  respond  to  the  situa¬ 
tion  in  many  different  ways,  each  one  provok¬ 
ing  a  different  modification  in  the  scene. 

Finally,  the  virtual  scenarios  were  tested  on¬ 
site,  during  a  two  sessions-evaluation  in  sev¬ 


eral  clubs.  Users  were  invited  to  judge  the 
environment  and  its  efficacy  through  a  24 
items  questionnaire  and  a  short  interview. 
The  evaluation  addressed  persuasiveness  of 
the  artifact,  presence  feeling,  quantity  and 
quality  of  concepts  learned,  user’s  perception 
of  consistency,  effectiveness,  and  usefulness 
of  the  product. 

Results 

In  general,  users  declared  to  have  more 
competence  on  drug-related  issues  and  risky 
behaviours  after  the  VE  experience.  They 
declared  to  understand  clearly  what  design¬ 
ers  want  to  communicate  with  them  (av. 
5.50/6  points),  to  perceive  a  strong  consis¬ 
tency  between  real  situations  and  virtual 
worlds  (av.  4.75/6  points)  and  to  consider  the 
VE  useful  to  educate  people  on  specific  drug 
effects.  Perceived  usefulness  of  the  product 
was  particularly  high  in  the  age  range  24-29 
for  alcohol  abuse  (x^  =10.814,p=.013  with 
d.f=4  a=.05). 

Conclusion 

Scenario-based-design  seems  to  be  a  pre¬ 
cious  methods  to  develop  VE  educational 
tools  for  risk  prevention.-  Playsafety  provides 
a  safe  educational  experience  to  young  peo¬ 
ple  who  potentially  could  be  involved  in  dan¬ 
gerous  contexts.  As  a  persuasive  technology 
working  on  cause-effect  relationships,  Play¬ 
safety,  effectively  communicates  what  risk 
underlies  many  common  situations  in  youth 
life. 

Novelty 

The  tool  addresses  the  lack  of  persuasive 
products  regarding  alcohol  and  safe-driving 
problems  in  young  people. 
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Exposure  in  vivo  is  a  well  tested  and  effective 
treatment  in  anxiety  and  phobic  disorders. 
However,  it  is  not  viable  to  expose  patients 
with  War  Post  Traumatic  Stress  Disorder 
(PTSD)  to  real  war  environments.  Being  vir¬ 
tual  reality  exposure  the  most  approximate 
simulation  of  a  real  war  situation,  we  hy¬ 
pothesized  that  treatments  of  War  PTSD  that 
included  virtual  reality  exposure  are  better 
than  the  current  forms  of  psychotherapeutic 
intervention,  namely  exposure  in  imagination. 
This  paper  describes  the  procedures  and 
presents  the  up  to  date  development  of  (VR) 
war  environments  as  a  methodology  to  im¬ 
plement  immersive  virtual  reality  VR  as  an 
alternative  psychotherapeutic  course  of  ac¬ 
tion  for  patients  with  the  diagnostic  of  War 
PTSD.  The  study  population  of  this  ongoing 
research  consists  of  male  subjects  with  the 
diagnostic  of  War  PTSD  according  to  DSM- 
IV-TR  (APA,  2001)  that  looked  for  treatment 
at  Julio  de  Matos  Hospital  in  Lisbon,  Portu¬ 
gal.  Participants  were  distributed  through  6 
treatments  plus  placebo  (VR;  Drug  treatment; 
Imagination  Exposure;  VR+Drug  Treatment; 
VR+lmagination  Exposure;  Drug  Treat- 
ment+lmagination  Exposure).  The  adequate 
therapeutic  dosage  with  Sertraline  (Zoloft, 
Pfeizer)  will  be  administrated  during  16 
weeks  to  the  Drug  Treatment  groups.  VR 
Treatment  groups  will  be  using  a  Head 
Mounted  Device  that  provides  an  immersive 
experience  on  the  following  war  virtual  sce¬ 
narios:  mine  deflagration,  assisting  casual¬ 
ties,  waiting  for  a  rescue  helicopter,  and  am¬ 
bush.  CAPS,  BDI,  STAI,  SCL-90,  MCM-ll  for 
Psychometric  measures  and  TAS,  DES,  PQ, 
SUDS,  heart  rate  and  blood  pressure,  ECG, 
EEG,  and  ACTH  for  VR  reactions  measures 
are  the  evaluation  procedures  selected  for 
assessing  the  results. 
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Virtual  Reality  (VR)  Exposure  has  proven  to 
be  efficacious  in  the  treatment  of  phobias. 
There  are  an  increasing  number  of  controlled 
studies  that  support  the  use  of  VR  as  an  ef¬ 
fective  tool  in  the  treatment  of  several  specific 
phobias  like  acrophobia,  claustrophobia,  spi¬ 
der  phobia,  and  flying  phobia  (i.e.,  Botella, 
Banos,  Villa,  Perpiha,  &  Garcia-Palacios, 
2000;  Emmelkamp  et  al.  2002;  Garcia- 
Palacios,  Hoffman,  Carlin,  Furness,  &  Botella, 
2002;  Rothbaum,  Hodges,  Anderson,  Price,  & 
Smith,  2002;  Wiederhold,  Jang,  Gevirtz,  Kim, 
Kim,  &  Wiederhold,  2002),  The  findings  sup¬ 
port  that  VR  exposure  is  as  effective  as  in 
vivo  exposure  and  some  of  the  studies  offer 
long-term  efficacy  (six  and  twelve  month  fol¬ 
low-ups).  Another  line  of  research  in  the  study 
of  treatment  effectiveness  is  to  investigate  the 
mechanisms  of  change  associated  to  the  suc¬ 
cess  of  VR  exposure.  The  data  regarding  this 
important  issue  are  very  scarce.  The  aim  of 
this  work  is  to  contribute  to  the  study  of  pre¬ 
dictors  of  outcome  in  the  field  of  VR  exposure 
treatments.  VR  has  some  unique  features  that 
may  be  involved  in  its  efficacy  and  can  play  a 
role  as  predictors  of  outcome.  One  important 
aspect  in  VR  treatments  is  the  degree  of  pres¬ 
ence  that  the  patient  feels  in  the  virtual  envi¬ 
ronment.  We  could  hypothesize  that  the  more 
presence  the  patients  feel  the  more  they  will 
be  involved  in  the  treatment  and  the  more 
effective  it  will  be.  Another  set  of  variables 
could  influence  treatment  response.  There  are 
some  psychological  differences  that  may  be 
relevant  to  the  VR  experience  like  hypnotiza- 
bility,  dissociation,  absorption,  imagery,  etc.  In 
this  work  we  are  interested  in  two  of  these 
variables:  Absorption  (the  tendency  to  be¬ 
come  involved  in  a  perceptual,  imaginative,  or 
ideational  experience)  and  dissociation  (the 
tendency  to  experience  disruptions  in  the  inte¬ 
grated  functions  of  consciousness,  memory, 
or  perception  of  the  environment).  The  aim  of 
this  work  is  to  study  the  possible  role  of  pres¬ 
ence,  absorption,  and  dissociation  in  the  treat¬ 
ment  response  to  VR  exposure  therapy.  The 
sample  was  composed  by  sixty  patients  meet¬ 
ing  DSM-IV  (APA,  2000)  criteria  for  different 
phobias  like  claustrophobia,  acrophobia,  in¬ 
sect  phobia,  and  flying  phobia  participating  in 
several  investigation  testing  the  efficacy  of  VR 


exposure. 
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Patients  frequently  report  severe  to  excruciat¬ 
ing  pain  during  medical  procedures.  The 
medical  literature  shows  post-hypnotic  sug¬ 
gestions  (PHS)  can  reduce  such  procedural 
pain,  and  recent  preliminary  clinical  studies 
suggest  that  virtual  reality  (VR)  distraction 
may  serve  as  an  even  more  powerful  non- 
pharmacologic  adjunct.  The  present  study  is 
the  first  to  examine  the  individual  and  com¬ 
bined  effects  of  post-hypnotic  suggestions 
(PHS)  and  VR.  In  the  present  experimental 
pain  study,  104  healthy  volunteers  partici¬ 
pated  in  a  double-blind  between-groups  de¬ 
sign  with  four  groups:  (No  PHS,  No  VR),  (No 
PHS,  Yes  VR),  (No  VR,  Yes  PHS),  and  (Yes 
PHS,  Yes  VR).  Each  subject  provided  sub¬ 
jective  0-10  ratings  of  cognitive,  sensory,  and 
affective  components  of  pain.  Afterwards 
subjects  received  a  Stanford  Hypnotizability 
test  to  measure  their  receptivity  to  hypnotic 
suggestions.  Immersive  VR  distraction  had 
robust  effects  on  pain  intensity,  pain  unpleas¬ 
antness,  and  time  spent  thinking  about  pain 
regardless  of  hypnotizability.  In  contrast, 
overall  post-hypnotic  suggestions  had  rela¬ 
tively  weak  effects  on  pain  intensity  and  un¬ 
pleasantness  but  had  robust  analgesic  effects 
for  participants  with  higher  hypnotizability 
scores.  For  highly  hypnotizable  subjects 
there  was  a  pattern  suggesting  the  possibility 
that  VR  +  PHS  combined  may  reduce  pain 
more  than  either  technique  alone.  Implica¬ 
tions  for  maximizing  pain  relief  are  discussed. 
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Background 

Since  2001,  researchers  have  developed  and 
tested  virtual  reality  (VR)  software  and  envi¬ 
ronments  across  a  variety  of  addictive 
drugs.  Research  in  addictions  has  primarily 
focused  on  testing  the  feasibility  of  VR  envi¬ 
ronments  to  elicit  cued-reactivity  (i.e.,  drug 
craving  and  physiological  responses).  Cue- 
reactivity  involves  exposing  substance  ad¬ 
dicted  persons  to  VR  drug  cues  and  VR  neu¬ 
tral  cues  while  assessing  subjective  craving 
and  physiological  reactivity.  Cue  reactivity  is 
based  on  the  theory  of  classical  conditioning 
where  drug  use  is  repeatedly  paired  with  en¬ 
vironmental  stimuli  and  these  stimuli  can  trig¬ 
ger  craving  and  reactivity.  Environmental 
stimuli  are  often  referred  to  as  cues  and  may 
consist  of  objects  (i.e.,  cigarettes,  drug  para¬ 
phernalia),  places  (e.g.,  crack  house,  bar), 
people,  and  scents  (i.e.,  smell  of  cigarettes  or 
beer).  Several  controlled  experimental  trials 
clearly  demonstrate  that  VR  drug  environ¬ 
ments  can  elicit  craving  and  physiological 
responses  compared  to  neutral  VR  environ¬ 
ments  in  nicotine  dependent  persons^'^  and 
cocaine  dependent  person.®  Currently  VR 
environments  have  been  developed  for  nico¬ 
tine,  cocaine,  and  aicohol.  The  marijuana  VR 
environment  is  being  constructed.  This  pres¬ 
entation  will  focus  on  current  results  and  the 
status  of  VR  environments  as  well  as  future 
directions  in  the  addiction  field. 

Methods 

All  studies  reviewed  and  presented  are  con¬ 
trolled  experimental  trials  with  substance  de¬ 
pendent  samples  using  strict  inclusion  and 
exclusion  criteria.  Across  trials,  participants 
are  exposed  to  both  VR  neutral  stimuli  and 
VR  drug  related  stimuli  using  a  VFX-3D  HMD 
(Interactive  Imaging,  Rochester,  NY)  con¬ 
nected  to  a  2Ghz  P-IV  PC.  VR  drug  cues 
consisted  of  both  inanimate  (i.e.,  drug  para¬ 
phernalia)  and  animate  cues  (i.e.,  social  set¬ 
tings  where  participants  are  offered  their  drug 
of  choice)  and  VR  neutral  cues  (i.e.,  non¬ 
drug  related  objects  or  scenes).  In  addition 
to  the  visual  cues,  recent  advances  in  our  lab 
have  led  to  the  addition  of  olfactory  cues  into 
the  VR  environments.  Current  data  on  olfac¬ 
tion  in  VR  cue  reactivity  wili  be  presented. 

Results 

Data  presented  will  focus  on  the  comparison 
of  drug  craving  and  autonomic  physiological 
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reactivity  between  VR  drug  cues  and  VR  neu¬ 
tral  cues.  A  summary  of  current  data  from 
nicotine,  alcohol,  and  cocaine  cue  reactivity 
trials  will  be  presented. 

Conclusions 

This  presentation  wiil  provide  an  overview  of 
VR  applications  in  addiction  research  and 
treatment.  Past,  current  and  future  directions 
will  be  discussed  along  with  lessons  learned 
to  date. 

Novelty/Discussion 

This  presentation  will  provide  the  first  over¬ 
view  of  VR  cue  reactivity  research  to  date, 
including  data  on  controlled  alcohol,  cocaine, 
and  nicotine  trials.  The  first  application  of  a 
novel  olfactory  stimuli  presentation  system  in 
VR  drug  research  will  also  be  high¬ 
lighted.  Future  research  and  treatment  appli¬ 
cations  will  be  discussed. 
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Research  Status:  Preliminary  clinical  trials 

Background 

Scent  is  one  of  the  most  salient  senses  for 
emotional  response.  The  olfactory  bulbs  pro¬ 
ject  into  nerve  tracts  that  synapse  into  the 
amygdala.  The  amygdala  is  connected  bidi¬ 
rectionally  to  the  hippocampus,  the  center  of 
many  memory  functions.  These  connections 
suggest  that  specific  smells  may  be  associ¬ 
ated  with  accessing  emotionally  laden 
memories.  Herz  &  Schooler^  and  Herz,^ 
have  investigated  the  associations  of  scent 
and  memory  providing  an  empirical  basis  for 
scent  research  in  virtual  reality  (VR).  In  re¬ 
spondent  conditioning,  scents  can  serve  as 
powerful  cues  that  trigger  physiological  and 
subjective  responses  to  past  conditioned 
events.  Building  upon  these  studies,  we  are 
exploring  the  use  of  scent  in  virtual  reality 
(VR).  The  addition  of  scent  will  extend  our 
previous  work  in  phobias^  and  enhance  our 
research  in  addictions'*  and  speech  pathol¬ 
ogy.^  This  presentation  outlines  the  process 
of  adding  scents  to  several  immersive  envi¬ 
ronments  being  tested  in  controlled  research 
trials. 

Method  &  Tools 

Scent  is  being  utilized  in  two  different  and 
related  areas.  First,  it  is  being  applied  to  fa¬ 
cilitate  immersion  in  VR  environments  as  an 
ambient  stimulus  in  a  job  interview  designed 
for  use  with  persons  who  stutter.  Second, 
scent  is  being  applied  directly  as  a  primary 
stimulus  to  facilitate  craving  responses  in 
controlled  trials  with  substance  abusing  and 
dependent  participants.  Examples  of  such 
implementations  using  the  EnviroScent  ma¬ 
chine  in  conjunction  with  Virtually  Better  VR 
environments  will  be  presented. 

Results 
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Initial  data  including  immersion  ratings,  and 
subjected  units  of  discomfort  (SUDs)  or  crav¬ 
ing  ratings  from  the  triais  will  be  discussed. 
Quaiitative  experiences  of  participants  in  trials 
will  be  discussed. 

Novelty 

To  our  knowledge  this  will  be  the  first  time 
that  our  Enviroscent  Machine  will  have  been 
used  with  immersive  VR  in  a  clinical  setting. 
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Reserach  Status:  Completed 

Several  functional  neuroimaging  studies 
have  studied  different  specific  aspects  of  hu¬ 
man  navigation. To  further  elucidate  re¬ 
gions  involved  during  performance  of  spatial 
navigation  and  to  determine  how  task  per¬ 
formance  influences  the  identified  brain  re¬ 
gions,  this  study  utilized  a  novel  virtual  reality 
(VR)  navigation  task  during  functional  mag¬ 
netic  resonance  imaging  (fMRI).  The  task 
enabled  both  learning  and  recall  of  routes  to 
be  assessed.  The  multivariate  Partial  Least 
Squares  analysis  (PLS)'*  was  used  to  identify 
time-varying  distributed  activity  patterns  in 
the  fMRI  data. 

Methods/Tools 

The  experiment  involved  navigation  through 
a  virtual  city  using  an  fMRI-compatible  virtual 
reality  platform.®  The  task  consisted  of  a  set 
of  3  learning  trials  for  Path  A  followed  by  1 
learning  trial  for  Path  B.  Each  learning  trial 
consisted  of  two  components:  passive  view¬ 
ing  of  the  path  by  video  playback,  and  active 
navigation  using  an  fMRI-compatible  Joystick. 
In  subsequent  short  (5  min)  and  long  (20 
min)  delay  recall  trials.  Path  A  was  navigated 
from  memory.  Twelve,  healthy,  right-handed 
subjects  participated.  Functional  MRI  experi¬ 
ments  were  performed  at  1.5T  (Signa,  GE 
Healthcare).  Behavioural  metrics  included 
time  to  completion  and  distance  travelled. 
For  the  primary  analysis,  learning  trials  were 
contrasted  with  recall  trials  (task-PLS)  follow¬ 
ing  motion  correction  and  temporal  detrend¬ 
ing.  Further  analysis  included  a  behavioural- 
PLS  in  which  time  to  completion  was  used  to 
determine  significant  brain-behaviour  correla¬ 
tions. 

Results 

One  significant  (p<0.01)  latent  variable  char¬ 
acterized  differences  in  brain  activity  be¬ 
tween  learning  trials  and  recall  trials.  Design 
scores  denoted  significant  differing  neuronal 
involvement  between  conditions.  Activation 
maps  indicated  that  bilateral  frontal  regions 


249 


Annual  Review  of  CyberTherapy  and  Telemedicine 


and  right  hippocampus  were  more  involved 
during  learning,  while  parietal  and  temporal 
regions,  such  as  the  left  parahippocampal 
gyrus,  bilateral  precuneus  and  cingulate  cor¬ 
tex,  were  more  active  during  recall.  A  second 
task-PLS  analysis  focused  on  the  4  learning 
trials,  identifying  a  pattern  related  to  the  first 
learning  trial  for  each  path.  Areas  involved  in 
initial  learning  included  right  precuneus  and 
the  cerebellum.  Subsequent  trials  recruited 
regions  surrounding  the  lateral  sulcus  bilater¬ 
ally  as  well  as  the  middle  and  medial  frontal 
gyri.  The  behaviour-PLS  confirmed  many  of 
these  findings  among  similar  brain  regions, 
identifying  patterns  that  were  related  to  gen¬ 
eral  task  performance,  to  initial  learning,  and 
that  specifically  distinguished  learning  from 
recall. 

Conclusion 

Findings  of  posterior  activations  during  recall 
are  consistent  with  the  notion  that  temporal 
regions  are  involved  in  spatial  information  re¬ 
trieval,  whereas  initial  encoding  of  a  complex 
route  demands  increased  precuneus  involve¬ 
ment.  The  predominant  activation  of  frontal 
and  temporal  regions  in  subsequent  learning 
trials  can  probably  be  associated  with  encod¬ 
ing,  maintenance,  and  recall  of  visual  cues, 
which  are  critical  for  successful  navigation. 

Novelty 

Combining  VR-fMRI  with  PLS  proved  to  be  a 
highly  sensitive  technique  for  examining  re¬ 
gions  engaged  during  spatial  navigation. 
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IMMUNE  ATTACK:  BUILDING  BIO¬ 
LOGICALLY  BASED  IMMUNE  SYSTEM 
SIMULATIONS  FOR  EDUCATION  AND 
TRAINING 
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Federation  of  American  Scientists 

Background 

A  growing  literature  suggests  that  simulation, 
visualization,  and  gaming  can  play  an  impor¬ 
tant  role  in  deepening  understanding  of  diffi¬ 
cult  concepts  in  mathematics,  engineering, 
and  science.  Learning  sciences  research 
suggests  that  learning  by  doing  with  under¬ 
standing  produces  better  transfer  than  mere 
doing  alone.''’^  Challenge-based  simulations 
can  provide  students  opportunities  to  receive 
feedback  and  revise  their  thinking,  a  critical 
part  of  the  learning  process.® Immune  At¬ 
tack  is  a  simulation  game  to  teach  biological 
concepts  related  to  immunology  and  wound 
infection,  allowing  the  student  to  explore  the 
internal  compartments  and  cells  of  the  hu¬ 
man  body  and  visualize  immunological  proc¬ 
esses. 

Method/Tools 

Immune  Attack  will  combine  3D  visualiza¬ 
tions  of  biological  structure  and  function  with 
advanced  educational  technologies  to  pro¬ 
vide  an  introduction  to  basic  concepts  in  im¬ 
munology  for  high  school  students.  It  Is  in¬ 
tended  to  be  as  fun  and  compelling  as  the 
computer  games  currently  played  by  many 
adolescents  and  young  adults.  8tudents  are 
motivated  with  a  series  of  progressively  more 
difficult  challenges  in  a  gaming  environment 
in  which  success  depends  on  increasingly 
sophisticated  grasp  of  concepts  in  immunol¬ 
ogy.  The  learning  experience  will  be  individu¬ 
alized  by  use  of  context-sensitive  help  and 
dialogues  and  continuous  assessment  tech- 
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niques  to  determine  when  the  learner  is  ready 
to  move  to  a  new  level. 

The  biological  models  are  being  developed 
working  closely  with  prominent  immunology 
researchers  and  educators.  Experienced 
video  game  developers  are  developing  the 
game  and  assisting  in  integrating  the  learning 
tools.  The  learning  objectives  and  instruc¬ 
tional  strategies  are  being  developed  in  con¬ 
sultation  with  biology  teachers,  at  both  the 
high  school  and  college  freshmen  levels,  and 
with  learning  research  scientists. 

Evaluation 

The  game  will  be  used  to  supplement  imunol- 
ogy  taught  as  a  part  of  biology  courses  given 
to  high  school  students  and  will  be  valuated  in 
high  school  biology  classes.  The  project  will 
be  evaluated  based  on  the  following  criteria; 
1)  Has  the  project  developed  biologically  cor¬ 
rect,  visually  compelling  simulations  of  the 
immune  system  that  can  be  easily  navigated 
by  people  without  specialized  technical  profi¬ 
ciency?  2)  Do  the  simulations  allow  revisions 
and  augmentations  and  encourage  use  of 
simulation  components  in  the  work  of  others? 
3)  Can  the  simulations  be  used  in  conjunction 
with  a  variety  of  assessment,  feedback,  and 
augmentation  tools?  4)  Does  use  of  the  game 
increase  students’  motivation  to  learn  the 
classroom  material? 

Novelty/Discussion 

Computer  games  hold  special  interest  to  a 
generation  who  has  grown  up  with  them,  and 
as  such,  they  show  promise  as  educational 
tools.  Whether  this  is  due  to  the  inherent 
challenge  built  into  game  play,  the  richness  of 
graphics  presented  to  the  user,  the  opportu¬ 
nity  to  interact  with  other  users  (in  web-based 
games),  the  story  or  context  in  which  the 
game  is  couched,  or  some  other  feature  is  an 
important  part  of  this  research  project.  Ex¬ 
ploiting  the  inherent  motivational  aspects  of 
games  and  simulations  for  education  and 
training  must  be  based  on  a  sound  under¬ 
standing  of  which  features  of  these  systems 
are  important  for  learning  and  why. 
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Research  Status:  MAMID  cognitive- 
affective  architecture  is  implemented;  Appli¬ 
cations  to  psychotherapy  research  are 
planned. 

Background 

VR  applications  in  psychotherapy  are  gaining 
prominence  in  the  research  community  and 
acceptance  among  practitioners.  However,  a 
promising  technology  remains  unexplored; 
computational  models  of  cognition  and  emo¬ 
tion. 

Cognitive  models  (also  termed  cognitive-  or 
agent-architectures)  aim  to  emulate  cognitive 
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processing  such  as  attention,  perception,  and 
decision-making,  and  are  used  by  cognitive 
scientists  to  advance  understanding  of  the 
mechanisms  and  structures  mediating  cogni¬ 
tion.  These  models  are  also  used  in  applied 
settings  to  improve  training  and  human- 
system  design.  Recently,  architectures  have 
been  developed  that  explicitly  represent  emo¬ 
tions:  both  emotion  appraisal  processes,  and 
effects  of  emotions  on  cognition. 

In  this  paper  I  first  describe  a  cognitive- 
affective  architecture  capable  of  modeling  the 
dynamic  generation  of  emotions  (affect  ap¬ 
praisal),  and  some  effects  of  emotions  on 
cognition.  I  then  describe  applications  of  this 
architecture  to  psychotherapy  research  and 
practice. 

Method  /  Tools 

The  MAMID  cognitive-affective  architecture 
dynamically  generates  four  emotions  (anxiety, 
anger,  joy,  sadness)  from  a  combination  of 
external  and  internal  stimuli  (e.g.,  incoming 
sensory  cues,  goals,  expectations).  The  archi¬ 
tecture  then  models  the  effects  of  these  emo¬ 
tions,  as  well  as  four  traits  (extraversion,  neu- 
roticism,  conscientiousness,  aggressiveness), 
on  the  cognitive  processes  and  structures 
mediating  decision-making  and  action  selec¬ 
tion.  For  example,  anxiety-linked  threat-bias 
is  modeled  by  biasing  attentional,  interpretive 
and  expectation-generation  processes  to  pref¬ 
erentially  process  threatening  stimuli,  and  de¬ 
rive  higher-threat  interpretations  and  expecta¬ 
tions.  MAMID  is  able  to  model  a  broad  range 
of  ‘stereotypes’  by  representing  emotion  ef¬ 
fects  in  terms  of  parameters  controlling  the 
architecture  structures  (memory)  and  proc¬ 
esses  (e.g.,  attention,  situation  assessment). 

Results 

Feasibility  of  the  model  was  demonstrated  in 
the  context  of  a  simulated  peacekeeping  sce¬ 
nario,  where  separate  instances  of  the  archi¬ 
tecture  controlled  the  behavior  of 
‘stereotypical’  unit  leaders  (‘anxious,’ 
‘aggressive,’  ‘normal’).  The  same  set  of  exter¬ 
nal  stimuli  triggered  distinct  emotions  in  the 
different  stereotypes,  and  their  effects  on  de¬ 
cision-making  then  caused  differences  in  ob¬ 
servable  behaviors.  The  MAMID  architecture 
is  domain-independent,  to  facilitate  transitions 
to  other  domains,  including  psychotherapy. 


Novelty  /  Discussion 

There  is  great  potential  for  application  of  cog¬ 
nitive-affective  models  in  psychotherapy  re¬ 
search  and  practice;  both  in  a  “stand-alone” 
mode,  and  coupled  with  synthetic  avatars,  to 
produce  more  realistic  virtual  characters. 

Research  MAMID  provides  a  tool  for  model¬ 
ing  the  etiology  and  treatment  of  a  variety  of 
disorders;  e.g.,  the  positive  feedback  cycle 
characterizing  generalized  anxiety  disorders 
and  phobias  could  be  modeled  by  represent¬ 
ing  the  increasing  predominance  of  anxiety- 
related  schemas  in  the  patient’s  long-term 
memory,  increased  sensitivity  to  anxiety- 
producing  stimuli,  and  generalization  across 
previously  neutral  stimuli.  Results  of  cogni¬ 
tive  restructuring  could  be  modeled  by 
‘exposing’  models  to  repeated  simulated  in¬ 
puts  representing  therapeutic  interventions  or 
changing  social  contexts,  and  modeling  the 
gradual  emergence  of  more  adaptive  mem¬ 
ory  schemas,  interpretations,  and  behavior. 
Improved  understanding  of  these  processes 
would  provide  opportunities  for  more  finely- 
tuned  assessment  and  treatment. 

Clinical  Practice  MAMID  provides  a  means  of 
controlling  the  behavior  of  avatars  in  thera¬ 
peutic  virtual  environments,  thereby  enhanc¬ 
ing  their  believability.  The  avatars  would  rep¬ 
resent  specific  stereotypes  (e.g.,  ‘aggressive 
audience  member,’  ‘shy  friend’),  allowing  the 
construction  of  virtual  social  situations  rele¬ 
vant  for  the  patient.  The  use  of  affectively- 
realistic  avatars,  embedded  within  simulated 
situations,  would  also  enhance  assessment. 
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TOWARDS  A  NEUROPSYCHOLOGI¬ 
CAL  BASIS  OF  PRESENCE 
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In  this  presentation,  we  will  review  recent 
neuropsychological  studies  regarding  tool 
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use  and  body  image  perception,  and  highlight 
their  importance  for  understanding  presence. 

The  experience  of  presence  appears  to  be  a 
complex  perception,  formed  through  an  inter¬ 
play  of  raw  multisensory  data,  spatial  percep¬ 
tion,  attention,  cognition,  and  motor  action,  all 
coupled  through  a  constant  dynamic  loop  of 
sensorimotor  correspondence.  Presence  re¬ 
search  studies  the  experience  of  being  in  a 
place  or  being  with  someone  as  it  is  mediated 
through  technology. 

The  perception  of  ourselves  as  part  of  a 
space  not  only  depends  on  a  passive  percep¬ 
tion  of  spatial  layout  but  also  on  the  ability  to 
actively  explore  an  environment,  allowing  the 
perceptual  systems  to  construct  a  spatial  map 
based  on  sensorimotor  dependencies.  By  in¬ 
corporating  telepresence  technology  that  sup¬ 
ports  our  bodily  perceptual  and  control  move¬ 
ments  as  part  of  the  ongoing  perceptual- 
motor  loop,  the  correlations  between  motor 
actions  and  multisensory  inputs  remain  intact 
and  a  sense  of  telepresence  or  ‘distal  attribu¬ 
tion’  may  occur. 

The  fact  that  technology  can  start  working  as 
a  transparent  extension  of  our  own  bodies  is 
critically  dependent  on  the  highly  plastic  na¬ 
ture  of  our  brain,  which  is  continuously  able 
and  prone  to  adapt  to  altered  sensorimotor 
contingencies.  This  fact  finds  its  basis  in  the 
significant  evolutionary  benefit  of  having  a 
negotiable  body  image  to  accommodate  life¬ 
time  development  and  change,  which  requires 
a  continuous  remapping  of  bodily  boundaries. 
A  radical  example  is  provided  by  the  amazing 
adaptation  processes  that  occur  in  the  body- 
image  of  people  with  one  or  more  lost  or  am¬ 
putated  limbs. ^  Although  body-image  adapta¬ 
tions  across  the  lifespan  can  afford  to  take 
their  time,  it  is  the  relative  speed  of  these  sen¬ 
sorimotor  adaptations  that  enables  us  to  ex¬ 
perience  man-made  technology  as,  quite  liter¬ 
ally,  part  of  ourselves  -  be  they  a  blind  per¬ 
son's  cane  or  an  advanced  telerobotic  arm. 

In  general,  the  space  that  surrounds  an  indi¬ 
vidual  can  be  meaningfully  segmented  into 
near  or  peripersonal  space  and  far  or  ex¬ 
trapersonal  space.  Animal  and  human  brain 
studies  have  confirmed  this  distinction,  show¬ 
ing  that  space  is  not  homogeneously  repre¬ 
sented  in  the  brain.^'^  Telepresence  technolo¬ 
gies  can  be  viewed  as  attempts  to  overcome 
the  boundaries  of  spatial  segmentation.  Their 
success  in  doing  so  is  evidenced  by  a  clinical 


case,  described  by  Berti  and  Frassinetti,'* 
where  a  patient,  after  a  right  hemisphere 
stroke,  showed  a  dissociation  between  near 
and  far  spaces  in  the  manifestation  of  ne¬ 
glect.  Using  a  line  bisection  task,  the  neglect 
was  apparent  in  near  space,  but  not  in  far 
space  when  bisection  in  the  far  space  was 
performed  with  a  projection  light  pen.  How¬ 
ever,  neglect  appeared  when  in  the  far  space 
bisection  was  performed  with  a  stick  (used  by 
the  patient  to  reach  the  line)  and  it  was  as 
severe  as  neglect  in  the  near  space.  Thus, 
this  study  provides  evidence  that  an  artificial 
extension  of  a  person’s  body  (the  stick) 
causes  a  remapping  of  far  space  as  near 
space  -  essentially  telepresence. 
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Backgrounds 

Patients  with  schizophrenia  show  a  tendency 
to  avoid  social  interaction  because  they  feel 
great  anxiety  during  their  social  interactions. 
Therefore,  they  have  trouble  making  social 
relationships.  This  trouble  is  largely  due  to  the 
fact  that  the  patients  have  emotional  with¬ 
drawal  as  well  as  passive/apathetic  social 
withdrawal.  These  social  deficits  of  patients 
has  been  overcome  through  social  skills  train¬ 
ing  programs,  which  provide  them  opportuni¬ 
ties  to  experience  previously  various  social 
situations  by  role  play  method. 

Objectives 

This  study  was  conducted  in  order  to  investi¬ 
gate  whether  an  interaction  with  virtual  ava¬ 
tars  can  evoke  patient’s  social  anxiety  and  the 
relationship  between  patient’s  symptom  se¬ 
verity  and  social  anxiety  traits. 

Method/Tools 

Fifteen  patients  with  schizophrenia  and  fifteen 
control  people  were  recruited.  A  male  and  a 
female  avatar  were  generated.  2x2x3  (group, 
avatar’s  gender  and  avatar’s  emotional  ex¬ 
pression)  was  used  in  this  experiment.  Each 
avatar  has  three  emotional  expressions, 
which  are  happy,  neutral,  and  angry.  Subjects 
performed  an  introduction  task  in  six  condi¬ 
tions  (gender  x  emotional  expression)  in  a 
random  order.  The  task  is  composed  of 
“approach,”  “listening  to  an  avatar’s  introduc¬ 
tion,”  and  “introducing  oneself  to  the  avatar.” 
After  all  six  tasks  were  performed,  subjects 
completed  a  State-Trait  Anxiety  Inventory 
(STAI)  questionnaire.  In  addition,  patient’s 
symptom  severity  was  evaluated  using  the 
Positive  and  Negative  Syndrome  Scale 
(PANSS). 

Results 

The  social  anxiety  level  to  avatar’s  emotional 
representation  was  significantly  different  in 
both  group  (p<.001).  In  control  group,  the 
level  was  lowest  when  subjects  coped  with  an 
avatar  expressing  “happy,”  while  the  highest 
anxiety  level  was  shown  when  they  faced  to 
an  avatar  expressing  “anger.”  However,  in 
patient  groups,  the  level  difference  was  not 
significant  between  “neutral”  and  “happy”  ava¬ 
tars.  Patients  only  showed  a  significantly  high 
anxiety  level  when  they  experience  “angry” 


avatar  conditions  compared  to  the  other  two 
conditions.  In  a  correlation  analysis  between 
patient’s  anxiety  levels  and  the  severity  of 
their  symptoms,  social  anxiety  on  “happy” 
and  “neutral”  avatars  was  positively  corre¬ 
lated  with  the  negative  syndrome  of  PANSS 
(happy  :  r=,539,  p=.038,  neutral  :  r=.533, 
p=.041).  Particularly,  the  anxiety  level  by 
“happy”  and  “neutral”  avatars  was  positively 
correlated  with  two  subscales  (N1:  blunted 
affect,  N4:  passive/apathetic  social  with¬ 
drawal)  of  the  negative  syndrome  of  PANSS 
(N1.  happy:  r=.549,  p=.034/  neutral:  r=.536, 
p=.039)  (N4.  happy:  r=.536,  p=.039/  neutral: 
r=.658,  p=.008). 

Conclusion 

Through  this  study,  we  found  that  virtual  ava¬ 
tar’s  emotional  expressions  could  cause  so¬ 
cial  anxiety  to  patients  with  schizophrenia 
and  that  patient’s  symptom  severity  was  cor¬ 
related  with  social  anxiety  during  the  time 
they  coped  with  avatars.  These  results  show 
that  the  more  severe  negative  symptoms  a 
patient  has,  the  higher  social  anxiety  they 
feel.  It  might  be  due  to  their  passive,  apa¬ 
thetic,  and  isolated  traits.  Therefore,  they 
tend  to  recognize  even  socially  positive  emo¬ 
tion  as  fearful  stimuli.  This  inference  could  be 
supported  by  the  relationship  between  anxi¬ 
ety  and  subscales  of  negative  symptom  of 
PANSS.  In  this  sense,  we  could  say  that  a 
virtual  avatar  could  provide  an  opportunity  for 
patients  to  experience  emotion  induced  so¬ 
cial  situations.  In  addition,  it  could  be  used 
for  training  them  to  cope  effectively  by  ex¬ 
periencing  emotions  close  to  reality  as  well 
as  to  find  out  the  clinical  characteristics  re¬ 
lated  to  patient’s  symptoms. 
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Research  Status:  Preliminary  clinical  trials 
(case  report)  currently  in  progress 

Background/Problem 

Cognitive  Behavioural  Therapy  (CBT)  with 
exposure  involves  having  patients  gradually 
remember  the  traumatic  event  and  process 
the  extreme  emotions  associated  with  that 
memory  so  they  can  eventually  remember  the 
traumatic  event  in  a  more  healthy  way.  CBT 
with  exposure  is  currently  the  treatment  of 
choice  for  Post  Traumatic  Stress  Disorder 
(PTSD).  There  is  strong  empirical  support  for 
the  efficacy  of  CBT  interventions  for  PTSD  for 
those  who  seek  and  complete  treatment.  Un¬ 
fortunately,  avoidance  of  remembering  the 
traumatic  event  is  a  defining  symptom  of 
PTSD,  and  the  majority  of  people  who  de¬ 
velop  PTSD  avoid  seeking  treatment.  Those 
needing  treatment  most  may  be  the  most 
likely  to  avoid  or  drop  out  of  traditional  treat¬ 
ment.  CBT  with  exposure  provides  patients 
with  opportunities  to  learn  to  control  their  own 
emotional  responses  when  confronted  with 
stimuli  that  elicit  physioiogical  and  emotional 
activation.  In  order  to  develop  new,  healthier 
patterns  of  thinking,  feeling,  and  acting  toward 
the  distressing  stimuli,  CBT  requires  the  prac¬ 
tice  of  new  beliefs  and  actions  in  the  presence 
of  stimuli  that  elicit  the  anxiety.  CBT  and  Vir¬ 
tual  Reality  (VR)  provide  patients  with  an  ideal 
context  for  gradually  remembering  memories 
of  the  traumatic  event.  VR  may  help  increase 
emotional  engagement  and  treatment  suc¬ 
cess.  VR  is  proving  effective  even  for  difficult 
patients  who  have  failed  to  respond  to  tradi¬ 
tional  therapy.^  Surveys  suggest  that  patients 
with  anxiety  disorders  are  much  more  recep¬ 
tive  to  VR  than  to  traditional  therapy,  perhaps 
because  VR  has  fewer  stigmas,  is  more  inter¬ 
esting,  and  less  threatening  than  traditional 
therapy. 

Method/Tools 

The  patient  we  treated  met  diagnostic  criteria 
for  PTSD  after  directly  witnessing  a  terrorist 
bus  bombing  attack  in  Haifa,  Israel,  and  re¬ 
ceived  a  9-visit  treatment  protocol  that  incor¬ 
porates  CBT+VR.  Among  other  things,  treat¬ 


ment  involves  viewing  scenes  of  the  terrorist 
bus  bombing  attack  through  a  position 
tracked  3-D  VR  heimet.  The  scenes,  com¬ 
puter  controlled  by  therapist  button  pushes, 
progress  in  severity,  culminating  in  a  bus 
pulling  up  to  a  bus  stop  in  an  Israeli  town, 
exploding  into  smoke  and  flame  with  loud 
sound  explosion  effects  (with  vibrotactile 
augmentation  via  amplified  speakers)  with 
accompanying  sounds  of  screaming  and  si¬ 
rens,  and  a  smouldering  limb  in  the  street. 
The  subject  completes  self-report  question¬ 
naires  and  is  assessed  prior  to  and  following 
the  treatment  by  two  experienced  clinicians 
(one  ciinician  blind  to  treatment  condition). 

Conclusion 

VR  exposure  therapy  is  a  promising  new  me¬ 
dium  for  treating  PTSD  from  terrorist  bus 
bombing  attacks. 

Novelty/Discussion 

There  are  currently  no  controlled  studies  in 
the  literature  on  the  topic  of  using  VR  expo¬ 
sure  therapy  for  treating  PTSD.  The  current 
project  addresses  a  compietely  original  re¬ 
search  topic  on  a  rapidly  growing  patient 
population  (using  VR  to  heip  treat  PTSD  from 
terrorism).  In  Israel,  there  have  been  dozens 
of  extremely  violent  deadly  terrorist  attacks 
involving  civilians  causalities  and  deaths 
(often  including  women  and  children)  in  the 
past  few  years  alone.  PTSD  is  a  very  chal¬ 
lenging  psychological  disorder  to  treat,  yet 
the  social  and  economic  cost  of  the  current 
trend  of  leaving  so  many  PTSD  victims  un¬ 
treated  is  unacceptable. 
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Geometry  education  has  proven  as  one  pow¬ 
erful  means  of  improving  spatial  abilities,  an 
important  component  of  human  intelligence. 
In  the  first  part  of  this  paper  we  summarize 
our  development  of  a  system  that  uses  col¬ 
laborative  augmented  reality  as  a  medium  for 
teaching,  and  uses  3D  dynamic  geometry  to 
facilitate  mathematics  and  geometry  educa¬ 
tion.  Our  immersive  collaborative  educational 
application,  specifically  developed  for  geome¬ 
try  education,  serves  as  the  basis  of  a  com¬ 
prehensive  evaluation  study  regarding  its  effi¬ 
cacy  in  training  spatial  abilities.  The  main  con¬ 
tribution  is  the  description  of  evaluation  de¬ 
sign  including  the  test  instruments,  learning 
tasks,  and  practical  experiences  with  using 
our  system  for  actual  training  of  high  school 
students.  Results  of  a  pre-study  with  spatial 
ability  tests  in  high  schools  are  presented. 
They  point  to  interesting  gender-specific  dif¬ 
ferences  of  strategies  when  solving  spatial 
ability  tests,  which  have  not  been  reported  in 
literature  before. 
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Virtual  Environments  (VEs)  offer  the  potential 
for  users  to  explore  social  situations  and  'try 
out’  different  behaviour  responses  for  a  vari¬ 
ety  of  simulated  social  interactions.  The  AS 
Interactive  project  was  a  three-year  pro¬ 
gramme  of  study  to  examine  the  use  of  vir¬ 
tual  environments  for  social  skills  training  of 
young  people  and  adults  with  autistic  spec¬ 
trum  disorders  (ASDs)  and  Asperger’s  Syn¬ 
drome  (AS).  The  project  research  team  com¬ 
prised  computer  scientists,  virtual  environ¬ 
ment  developers,  human  factors  researchers, 
psychologists,  and  autism  training  specialists 
working  with  individuals  from  AS  social 
groups  and  students  and  teachers  from  au¬ 
tism  specialist  schools. 

The  first  consideration  of  the  project  was  the 
feasibility  of  constructing  usable  social  envi¬ 
ronments  using  virtual  reality  technology. 
Two  types  of  virtual  environments  were 
evaluated;  single  user  virtual  environments 
(SVEs)  and  collaborative  virtual  environ¬ 
ments  (CVEs).  In  the  SVE  the  user  was 
guided  through  a  social  interaction  task  and 
invited  to  make  choices  about  what  to  do  and 
what  to  say  in  situations  represented  within  a 
virtual  cafe  and  a  virtual  bus.  In  the  CVE 
several  users,  positioned  at  different  PCs, 
simultaneously  shared  the  same  virtual  envi¬ 
ronment,  representing  a  virtual  cafe  or  an 
interview  scenario.  The  intended  use  of  the 
CVE  was  that  a  teacher  or  training  advisor 
could  support  the  user  by  taking  the  role  of 
one  of  the  other  characters  and  providing 
guidance  using  virtual  social  interaction. 

Observation  studies  found  that  SVEs  were 
much  more  successful  than  CVEs  as  the  lat¬ 
ter  was  too  complex,  technologically,  to  set 
up  in  school  and  users  tended  not  to  engage 
with  other  avatars  in  conversation.  SVEs 
however  could  be  used  in  one  to  one  tuition, 
which  gauged  the  suitability  of  the  program 
for  individual  students  and  identifying  their 
level  of  understanding  or  in  group  exploration 
and  reflection  (where  a  teacher  led  a  discus¬ 
sion  with  a  small  group  of  students  viewing 
the  VE  on  a  projected  screen)  which  was 
useful  for  identifying  social  cues  in  the  VE 
and  understanding  the  perspectives  of  oth¬ 
ers. 

The  conclusions  drawn  from  this  project  were 
that  successful  VEs  for  teaching  social  skills 
to  people  with  Asperger’s  Syndrome  (AS), 
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will  need  sufficient  scaffolding  inbuilt  to  sup¬ 
port  learning  whilst  also  having  the  flexibility 
to  adapt  for  unforeseen  problems  through 
teacher  intervention.  Allowing  a  teacher  to 
individualise  the  content  of  the  VEs  in  some 
way  is  important,  as  it  shouid  not  be  assumed 
that  a  student  has  understood  the  lesson  by 
appearing  to  exhibit  the  appropriate  behaviour 
for  the  given  task.  Some  behaviour  might  be 
individual  to  that  user  and  only  someone  who 
knows  the  user  will  understand  this.  This  pa¬ 
per  aims  to  discuss  the  issues  behind  the  de¬ 
sign  and  development  of  the  SVEs  and  the 
features  which  can  be  used  to  faciiitate  the 
teaching  of  social  skills  to  people  with  AS, 
who  have  very  individualised  needs. 
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Background/Problem 

Patients  with  schizophrenia  usually  lack  social 
skills  and  have  an  inability  to  communicate 
effectively  with  people,  confirm  and  express 
their  feelings,  and  understand  interpersonal 
boundaries.  They  may  soive  their  problems  in 
an  unsuitable  manner  or  they  may  have  few 
solutions.  In  this  study,  we  developed  the  Vir¬ 
tual  Reality  system  to  measure  social  problem 
solving  ability  according  to  the  state  of  illness 
of  the  patient  with  schizophrenia. 


Method/Tools 

We  developed  a  project-based  VR  system 
because  schizophrenic  patients  feels  very 
anxious  when  wearing  a  Head-mounted  dis¬ 
play.  Narrative-based  contents  were  con¬ 
structed  to  induce  schizophrenic  patients  into 
proper  goais  to  assess  social  problem  solv¬ 
ing  abilities.  Survey  for  selecting  8  compli¬ 
cate  social  problems  among  many  daily¬ 
happening  social  problems  and  setting  their 
difficulty  was  conducted  for  50  normal  peo¬ 
ple.  The  virtual  environment  and  virtual  ava¬ 
tar  matched  with  8  complicated  situations 
were  built  by  using  3D-MAX  and  were  con¬ 
verted  into  MAP  or  MDL  format  for  rendering 
in  A6  engine.  The  behavioural  database 
which  consists  of  15  avatars,  40  apparels,  70 
actions  and  6  faciai  expressions  was  con¬ 
structed  to  make  a  flexible  and  dynamic  ava¬ 
tar.  Aiso  eye-blinking  and  lip-synching  was 
roughly  implemented  to  make  the  virtual  ava¬ 
tar  more  realistic  and  live-like.  The  VR  sys¬ 
tem  was  designed  to  give  patients  the  infor¬ 
mation  that  is  needed  in  problem  solving  be¬ 
cause  we  want  to  measure  the  pure  social 
problem  solving  ability  except  cognitive  as¬ 
pects  such  as  memory.  It  was  implemented 
by  making  an  information  window  in  which 
patients  know  what  time  it  is,  how  much 
money  I  have,  today’s  schedule,  and  what  I 
should  do.  After  VR  experience  in  each  con¬ 
tent,  Q&A  panels  about  the  complicated 
situation  appeared  and  patients  could  select 
their  own  solution  about  given  social  prob¬ 
lems  by  using  an  8-button  joystick. 

Results/  Conclusion 

In  this  study,  we  designed  a  VR  system  to 
assess  schizophrenic's  social  problem  solv¬ 
ing  ability  except  the  cognitive  ability  of 
schizophrenia.  Reaction  time  responding  to 
the  Q&A  panel  and  problem  solution  in  given 
social  VR  situation  is  extracted  from  the  pro¬ 
posed  VR  system  to  assess  social  problem 
abilities.  After  having  experiences  in  session, 
schizophrenia  answers  following  questions: 
computer  experience  scale,  immersive  ten¬ 
dencies  questionnaire,  virtual  reality  ques¬ 
tionnaire,  sociai  problem  solving  index,  posi¬ 
tive  and  negative  syndrome  scale,  KWIS. 
Now  we  are  gathering  data  from  associated 
hospital  and  have  a  plan  to  validate  usabiiity 
and  effectiveness  of  proposed  VR  contents 
by  showing  that  VR  parameters  are  closely 
correlated  with  the  traditional  assessment 
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tools  e.g.  SPSI. 

Novelty/Discussion 

In  this  study  we  develop  a  VR  system  for  pa¬ 
tients  with  schizophrenia  to  perceive  more 
realistic  and  dynamic  situations,  which  could 
be  difficult  when  constructed  from  other  media 
such  as  text,  pictures,  or  video.  However  ad¬ 
ditional  research  of  the  usability  and  validation 
of  the  proposed  VR  system  is  needed  and  is 
being  conducted  in  our  lab. 

Contact: 

Jaehun  Kim 

Department  of  Biomedical  Engineering 

Hanyang  University 

Email:  jaehunkim@bme.hanyang.ac.kr 

Presenter:  Kwanguk  Kim  MS 

INVESTIGATION  OF  SOCIAL  CUE  PER¬ 
CEPTION  IN  SCHIZOPHRENIA  USING 
VIRTUAL  REALITY 

Kwanguk  Kim,  M.S.\  Jaehun  Kim,  M.S.\ 

Daeun  Park,  B.S.\  Jeonghun  Ku,  Ph.D.\  Hee 
Jeong  Jang,  B.A.^;  Sung  Hyouk  Park,  M.D.^; 

So  Young  Kim,  B.A.^;  Chan  Hyung  Kim,  M.D.' 
Ph.D^;  Jae-Jin  Kim,  M.D.,  Ph.D^;  Jang  Han 
Lee,  Ph.D.^;  In  Young  Kim,  M.D.,  Ph.D.\  and 
Sun  I.  Kim,  Ph.D’ 

'  Department  of  Biomedical  Engineering,  Hanyang 
University 

^  Institute  of  Behavioral  Science  in  Medicine,  Yon- 
sei  University  Severance  Mental  Health  Hospital 

Background/Problem 

Impairment  in  social  skills  is  one  of  the  few 
criteria  that  all  individuals  diagnosed  with 
schizophrenia  must  meet.  Successful  social 
skill  requires  the  coordination  of  many  skills, 
including  social  perception  that  involve  the 
decoding  and  interpretation  of  social  cues  in 
others.  Previous  research  on  the  social  per¬ 
ception  in  schizophrenia  has  focused  on  the 
interpretation  of  emotion  from  facial  expres¬ 
sions.  Recently,  there  have  been  reports  on 
the  potential  of  virtual  reality  (VR)  in  social 
skills  training.  In  this  study,  we  developed  VR 
for  social  perception  assessment  including  the 
interpretation  of  non-facial  expressions.  And 
we  hypothesized  that  schizophrenic  subjects 
would  perform  significantly  more  poorly  than 
normal  subjects  on  the  VR  test. 


Method/Tools 

For  the  effective  sociai  perception  test,  VR 
may  be  needed  to  compose  various  situa¬ 
tions  and  contents.  So,  we  used  VR  Behav¬ 
iour  &  Facial  Data  Base  architecture 
(VRBFDB)  for  easy  and  fast  VR  composition. 
This  architecture  is  composed  of  predefined 
93  behaviours  and  7  faciai  expressions  data 
base.  In  this  data  base,  we  selected  behav¬ 
iours  and  facial  expressions  according  to 
contents.  And,  VR  of  35  contents  is  devei- 
oped.  The  contents  can  be  divided  into  two 
sub-contents  "Attention  to  and  interpretation 
of  relevant  cue  test"  and  "Emotion  Recogni¬ 
tion  test."  "Attention  to  and  interpretation  of 
relevant  cue  test"  is  composed  of  non-verbal 
social  situation,  recognition,  and  verbai  sociai 
situation  recognition,  and  "Emotion  Recogni¬ 
tion  test"  is  composed  of  happy,  sad,  angry, 
and  surprise  facial  expression  recognition. 

The  21  contents  are  selected  from  35  con¬ 
tents  based  on  70%  selection  of  normal  sub¬ 
jects.  The  subjects  consisted  of  17  patients 
(12  males  and  5  females)  and  19  controls  (12 
males  and  7  females).  This  study  measured 
VR  data,  questionnaire  data,  and  symptom 
data.  VR  data  included  participant’s  response 
results,  reaction  time,  presence,  and  experi¬ 
ence  of  computer.  Questionnaire  data  in¬ 
cluded  age,  education,  gender,  intelligence 
(K-WAIS)  and  so  on.  Symptom  data  is  meas¬ 
ured  by  PANSS  (Positive  and  Negative  Syn¬ 
drome  Scales). 

Results/Conclusion 

According  to  analyzed  results,  schizophrenic 
subjects  perform  significantly  more  poorly 
than  normal  subjects  on  the  VR  test 
(difference  0.86).  In  detail,  the  difference  is 
0.90  in  "Attention  to  and  interpretation  of 
relevant  cue  test"  and  0.82  in  "Emotion  rec¬ 
ognition  test."  One  result  shows  that  emotion 
recognition  test  corresponds  with  established 
research  on  schizophrenia  on  emotional  fa¬ 
cial  recognition.  And,  the  other  result  shows 
that  relevant  cue  test  is  showing  that  schizo¬ 
phrenia  also  has  a  difficulty  to  recognize  so¬ 
cial  cues. 

Novelty/Discussion 

In  this  study,  we  developed  and  used  Virtual 
Reality  Behavior  &  Facial  Data  Base 
(VRBFDB)  for  saving  developing  times  and 
costs.  And  we  apply  VR  to  finding  the  differ- 
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ence  between  schizophrenic  subjects  and 
normal  subjects  in  social  cue  perception. 

Contact: 

Kwanguk  Kim  M.S. 

Department  of  Biomedical  Engineering 
Hanyang  University 

17  HAENG-DANG  DONG  SEONG-DONG  GU 

Seoul  133-605,  Korea 

Phone:  +82-2-2290-0693 

Fax:  +82-2-2296-5943 

Email:  kwana6@bme.hanvana.ac.kr 
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THE  RELATIONSHIP  BETWEEN  INTER¬ 
PERSONAL  DISTANCE  AND  SYMP¬ 
TOMS  OF  SCHIZOPHRENIA  ACCORD¬ 
ING  TO  DIFFERENT  EMOTIONAL 
CATEGORIES  USING  VIRTUAL  REAL¬ 
ITY 

Sung  Hyouk  Park’^,  M.D.,  So  Young  Kim^, 

B.A. 

Jeonghun  Ku^,  Ph.D.,  Hee  Jeong  Jang^,  B.A., 
Won  Yong  Hahn^,  M.S.,  In  Young  Kim  ,  M.D., 
Ph.D.,  Sun  I.  Kim^  Ph.D.,  Chan-Hyung  Kim^’^ 
M.D.,  Ph.D.,  Jae-Jin  Kim^  ^  M.D.,  Ph.D. 

^Department  of  Psychiatry,  Yonsei  University  Col¬ 
lege  of  Medicine 

^Mind  mapping  laboratory,  Institute  of  behavioral 
science  in  medicine, 

Yonsei  University  College  of  Medicine 
^Department  of  Biomedical  Engineering,  Hanyang 
University,  Seoul,  Korea 

Schizophrenia  is  a  disease  characterized  by  social 
impairment.  Adequate  interpersonal  distance  is 
important  for  good  interpersonal  relationship  and 
for  the  social  rehabilitation  of  schizophrenia.  There 
have  been  few  researches  about  interpersonal 
distance  of  schizophrenia.  We  tried  to  improve  the 
method  of  presenting  stimuli  through  virtual  reality. 
We  constructed  virtual  environment  and  virtual 
avatar  with  gradual  facial  expressions.  Fifteen 
schizophrenic  patients  and  control  subjects  partici¬ 
pated  in  this  experiment.  This  experiment  is  con¬ 
sisted  of  6  blocks.  In  each  block,  subjects  were 
asked  to  approach  the  avatar  with  gradual  facial 
expressions  representing  one  emotional  category, 
to  the  most  comfortable  place  and  introduce  them¬ 
selves.  We  measured  the  distance  subjects  took 
from  the  avatars.  We  explored  the  characteristics 
of  interpersonal  distances  of  schizophrenia  and 
the  relationship  among  the  distances,  symptoms  of 
schizophrenia  and  emotional  categories  (happy, 
neutral  or  angry).  There  were  major  effects  of 
group  and  emotion  on  distances.  There  were  an 


inverse  correlation  between  the  distances  from 
avatars  with  angry  emotion  and  negative  symp¬ 
toms.  These  results  suggest  that  negative  symp¬ 
toms  of  schizophrenic  patients  may  have  an  asso¬ 
ciation  with  deceit  of  social  skills  and  that  negative 
symptoms  of  schizophrenia  may  have  effects  on 
processing  of  emotions,  especially  of  negative 
valence. 

Contact: 

Sung  Hyouk  Park 
ERiail:  inuone@hanmail,net 
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VIRTUAL  REALITY  THERAPY  FOR 
SOCIAL  PHOBIA:  ITS  EFFICACY 
THROUGH  A  CONTROL  STUDY 

Evelyne  Klinger,  ENG.,  PhD.candidate^  ®, 
Stephane  Bouchard  Ph.D.,  Patrick 
Legeron,  M.D.^,  Stephane  Roy,  Psy.^, 
Francoise  Lauer,  Psy.'*,  Isabelle  Chemin, 

M.A.  ,  and  Pierre  Nugues,  Ph.D.® 

’  ENSICAEN-GREYC,  Caen,  France 
^  University  of  Quebec  in  Outaouais,  Gatineau, 
Quebec,  Canada 

®  CBT  Unit,  Sainte-Anne  Hospital,  Paris,  France 
''stimulus,  Paris,  France 
®  LTH  Department  of  Computer  Science,  Lund 
University,  Sweden 

®  GET/ENST-Paris,  CNRS/LTCI,  Paris,  France 

Research  status:  Completed  control  study 

The  efficiency  of  virtual  exposure  for  phobic 
disorders  has  been  confirmed  by  several 
studies,  especially  in  the  case  of  the  fear  of 
public  speaking  (FOPS).  However,  FOPS  is 
a  less  severe  subtype  of  social  phobia  and 
does  not  reflect  entirely  the  diversity  of  social 
situations  that  could  potentially  induce  anxi¬ 
ety.  In  an  earlier  study,  we  reported  a  virtual 
reality  (VR)-based  clinical  protocol  designed 
to  assess  the  efficacy  of  a  Virtual  Reality 
Therapy  (VRT)  on  a  larger  set  of  social  pho¬ 
bias  compared  to  a  validated  psychological 
treatment  (group-CBT=  Cognitive  Behavior 
Therapy).  The  virtual  environments  (VEs) 
used  in  the  treatment  recreate  four  situations 
dealing  with  social  anxiety:  performance,  inti¬ 
macy,  assertiveness,  and  scrutiny.  With  the 
help  of  the  therapist,  the  patient  learns 
adapted  cognitions  and  behaviors  in  order  to 
reduce  anxiety  in  the  corresponding  real 
situations. 
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The  study  is  based  on  a  pre-post  design 
where  the  control  condition  is  a  gold-standard 
treatment.  Thirty-six  participants,  diagnosed 
with  social  phobia  and  meeting  the  inclusion 
and  exclusion  criteria,  were  matched  on  either 
VRT  or  group-CBT.  Both  treatments  lasted 
twelve  weekly  sessions  and  were  delivered 
according  to  a  treatment  manual  in  the  pres¬ 
ence  of  a  CB-therapist.  In  VRT  condition,  ses¬ 
sions  were  individual  and  lasted  about  45 
minutes,  with  less  then  20  minutes  of  expo¬ 
sure  to  the  four  VEs  either  for  assessment  or 
therapy,  while  in  group-CBT  condition  two 
hours  sessions  were  delivered  with  about 
eight  people,  a  format  which  allows  creating 
multiple  social  situations.  In  both  cases,  fur¬ 
ther  exercises  were  prescribed  in  vivo  appli¬ 
cation  of  the  principles  developed  during  the 
sessions.  The  participants  were  submitted  to 
“pre-post”  assessment  of  the  full  spectrum  of 
social  phobia,  from  key  symptoms  to  global 
functioning. 

Using  repeated  measures  ANOVAs  (2  Condi¬ 
tions  X  2  Temps),  three  families  of  hypotheses 
were  tested:  social  phobia  (as  measured  with 
LSAS  score  and  subscores),  social  function¬ 
ing  (assertiveness  -  Rathus  and  SCIA),  and 
general  functioning.  Given  these  hypotheses 
and  the  use  of  a  gold-standard  control  condi¬ 
tion,  the  results  include  the  effect-sizes  of  the 
Condition  by  Time  interactions  and  the  esti¬ 
mates  these  differences’  importance.  They 
indicate  that  both  treatments  were  highly  ef¬ 
fective  to  reduce  social  anxiety  and  social 
avoidance  and  to  improve  social  as  well  as 
global  functioning,  with  no  significant  differ¬ 
ences  between  the  two  treatments  except 
assertiveness. 

Our  results  clearly  show  the  efficacy  of  virtual 
reality  in  the  treatment  of  social  phobia.  They 
also  demonstrate  that  people  can  react  emo¬ 
tionally  to  virtual  humans  and  their  behaviors, 
even  if  they  are  unrealistic  representations  of 
humans.  A  promising  explanation  can  be 
found  in  the  interplay  between  believability  of 
a  virtual  reality  and  emotions  felt  by  the  user 
immersed  in  the  virtual  environment. 

These  statistical  analyses  represent  a  new 
step  in  the  use  of  our  collected  data,  and  in 
the  comprehension  of  emotional  contexts, 
interesting  features  of  our  research  are  the 
selection  of  a  sample  composed  of  young 
adults  showing  clinically  significant  and  lasting 
social  phobia,  the  use  of  VEs  that  tackle  di¬ 
verse  social  situations,  and  the  assessment  of 


the  full  spectrum  of  social  phobia. 

References 

North,  M.,  North,  S.,  and  Coble,  J.R.  (1998). 
Virtual  Reality  Therapy  :  An  effective  Treat¬ 
ment  for  the  Fear  of  Public  Speaking.  Inter¬ 
national  Journal  of  Virtual  Reality.  3  (2):2-6. 
Roy,  S.,  Klinger,  E.,  Legeron,  P.,  Lauer,  F., 
Chemin,  I.,  and  Nugues,  P.  (2003).  Defini¬ 
tion  of  a  VR-based  protocol  to  treat  social 
phobia.  Cyberpsychology  &  Behavior.  6 
(4);4 11-420. 

Contact: 

Evelyne  Klinger 
43,  rue  Appert 
44100  Nantes  France 
Tel:  +33  2  40  69  59  21  (home) 

Fax:  +33  2  40  69  59  21  (home) 

Email:  klinqer@enst.fr 


Presenter:  Manfred  Krapp  MD  PhD 

THE  COMPUTER  AS  A  HELPMATE 
FOR  IMAGINATION 

Dr.  med.  Manfred  Krapp 

C.G.  Jung-lnstitute  Zuerich 

This  paper  deals  with  virtual  reality  (VR)  and 
Cyberspace  from  a  psychoanalytical  point  of 
view,  especially  regarding  the  analytical  psy¬ 
chology  of  C.G.  Jung.  Electronic  media  have 
basically  changed  the  relationship  between 
the  Ego  and  the  unconscious.  The  discourse 
of  electronic  media  is  right-hemispherical  and 
imaginal  and  corresponds  to  the  primary 
process  of  the  unconscious.  Flusser  empha¬ 
sises  a  new  faculty  of  imagination,  "computer 
images  simulate  brain  processes.  The  im¬ 
ages  that  form  there  are  almost  directly  pro¬ 
jected  from  the  brain  to  the  outside,"  they  are 
"dreams  made  exact."  This  contrasts  to  the 
secondary  process  of  the  Ego  which  is  char¬ 
acteristic  of  the  left  hemisphere  of  the  brain 
and  of  alphabetic  writing  and  print  with  its 
linear-perspectival  discourse  (Me  Luhan). 

The  electronic  media  has  relativised  time  and 
space  in  the  same  way  as  the  border  be¬ 
tween  individuals  and  between  ego  and  un¬ 
conscious  which  become  more  permeable 
and  flexible. 

Based  on  this  point  of  view  I  will  reflect  on 
the  possibilities  and  limits  of  VR  and  Cyber- 
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space.  The  space  of  inner  images  can  be  rep¬ 
resented  very  well  by  the  means  of  hypertext 
with  its  manifold  refer  ability  and  simultaneity. 
This  is  exemplified  by  a  borderline-patient's 
initial  imagery  which  are  analysed  by  the 
means  of  the  computer-interpretation-support 
system  ATLAS/ti.  It  is  based  on  the  proceed¬ 
ings  of  Grounded  Theory  and  was  specifically 
developed  for  the  requirements  of  qualitative, 
hermeneutic  research.  It  is  based  on  semantic 
network  representations  and  realizes  in  this 
way  the  hypertext  structure. 

Referring  to  this  case  the  capacity  of  cy¬ 
bertherapy  is  discussed,  how  it  could  function 
like  art  therapy  in  a  broader  sense  fostering 
the  emergence  of  inner  images,  and  symbols 
with  their  creative  potential  and  self-healing 
forces.  In  analytical  psychology  and  art  ther¬ 
apy  active  participation  on  creating  individual 
metaphors,  images  and  symbols  is  necessary 
in  order  to  strengthen  the  ego  and  to  restore 
the  capacity  of  symbolisation  and  imagination. 
Cybertherapy  could  be  a  therapeutic  tool  for 
concretizing  and  structuring  the  primary  proc¬ 
ess  material  as  it  emerges  in  analytical  treat¬ 
ment  offering  a  symbolic  and  archetypal  im¬ 
agery  for  the  patient. 
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Background 

A  virtual  avatar  has  been  used  for  various 
applications  that  require  communication 
among  persons,  or  to  train  or  educate  people 
by  demonstrating  human-like  behavior.  Re¬ 
cently,  much  research  has  shown  an  en¬ 
hancement  of  virtual  avatar  technology,  with 
the  avatar  perceived  as  a  real  person.  As  a 
result,  the  technology  could  begin  to  be  used 
for  observing  human  behavior  to  a  virtual 
avatar. 

Objectives 

This  paper  concerns  whether  a  virtual  avatar 
could  be  perceived  as  a  real  human  by  pa¬ 
tients  with  a  mental  illness,  particularly 
schizophrenic  patients,  as  well  as  whether  a 
virtual  avatar  could  acquire  patients’  behav¬ 
ioral  characteristics  during  a  short  conversa¬ 
tion. 

Method/Tools 

For  this,  we  designed  a  virtual  avatar  that 
was  standing  in  a  virtual  room,  with  eleven 
schizophrenic  patients  assigned  the  task  of 
approaching  the  virtual  avatar,  initiating  a 
conversation,  and  providing  answers  to  the 
avatar’s  questions.  To  measure  behavioral 
parameters  in  the  virtual  environment,  we 
acquired  the  interpersonal  distance  and  the 
verbal  response  time.  In  addition,  we  rated 
patients  on  the  Positive  and  Negative  Syn¬ 
drome  Scale  (PANSS)  in  order  to  investigate 
a  relationship  between  patients’  symptomatic 
characteristic  and  behavioral  parameters. 

Results 

Results  of  this  study  revealed  that  the  inter¬ 
personal  distance  was  negatively  correlated 
with  the  negative  syndrome  scale,  which  is  a 
subscale  of  PANSS  (r=-0.687,  p=0.02).  By 
contrast,  the  verbal  response  time  was  not 
correlated  with  any  other  subscale  of 
PANSS.  However,  after  analyzing  this  vari¬ 
able  with  sub-items  of  the  negative  syndrome 
of  PANSS,  two  positive  correlations  were 
found:  one  with  blunted  affect  (r=.638, 
p=.035)  and  the  other  with  poor  rapport 
(r=.615.  p=.044). 

Discussion 
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The  negative  correlation  between  the  distance 
and  negative  symptom  severity  observed  in 
this  study  could  represent  that  they  keep 
closer  distance  during  interacting  with  an  ava¬ 
tar  due  to  patient’s  symptom  severity.  There¬ 
fore,  it  also  could  be  said  that  the  severer 
negative  symptom  patients  have,  the  harder 
they  recognize  or  interact  with  the  other  ex¬ 
pressing  emotion  in  a  social  interaction.  This 
result  is  consistent  with  previous  studies  that 
investigated  the  relationship  between  schizo¬ 
phrenic  patients’  interpersonal  distance  and 
their  symptoms.  And.  the  positive  correlation 
between  the  verbal  response  time  acquired 
and  subscales  in  PANSS  in  this  study  could 
be  explained  by  the  definition  of  these  sub¬ 
scales.  The  blunted  affect  score  represents 
the  amount  of  emotional  change,  which  is 
characterized  as  the  reduction  of  facial  ex¬ 
pression,  emotional  modulation,  and  gesture 
during  communication.  By  contrast,  the  poor 
rapport  score  represents  the  amount  of  inti¬ 
macy  to  an  interviewer  or  the  level  of  deficit  in 
forming  a  relationship,  which  is  characterized 
as  distancing  a  human  relationship  and  the 
reduction  of  verbal  and  nonverbal  communi¬ 
cation.  Therefore,  this  positive  correlation 
means  that  the  less  a  patient’s  emotional  re¬ 
sponse,  intimacy,  and  relationship  making 
with  a  virtual  avatar,  the  slower  they  answer 
the  avatar’s  question.  Inferring  from  these 
results,  we  conclude  that  the  virtual  avatar 
could  be  perceived  as  a  real  human  by 
schizophrenic  patients,  the  avatar  could  facili¬ 
tate  the  schizophrenic  patients’  behavioral 
characteristics,  and  the  avatar  could  be  used 
as  a  tool  for  assessing  the  behavioral  charac¬ 
teristics  of  patients  with  schizophrenia. 
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Background 

Facial  expression  research  has  been  issued 
on  areas  such  as  social  psychology,  psychia¬ 
try,  and  so  on.  Particularly,  patients  with 
mental  illness  showed  deficits  in  reading  and 
recognizing  facial  expression,  so  that  they 
suffer  from  their  social  life  due  to  the  deficit. 
These  studies  mostly  focused  on  facial  rec¬ 
ognition  ability  and  were  conducted  with  still 
images  of  people.  However,  the  still  images 
as  stimuli  have  limitations,  which  are  lack  of 
controllability  and  flexibility,  and  lack  of  reality 
due  to  the  inability  to  give  a  dynamic  image. 
Actually,  dynamic  representation  of  facial 
expressions  is  necessary  because  it  is  more 
effective  to  be  recognized. 

Objectives 

The  objectives  of  this  study  were  to  propose 
a  method  in  order  to  present  dynamic  facial 
expressions  having  various  intensity,  to  in¬ 
vestigate  human’s  perception  to  avatar’s  fa¬ 
cial  expressions  divided  by  gradual  strength, 
to  investigate  how  the  perception  varies  to 
the  strength  of  facial  expression,  as  well  as 
to  investigate  how  different  the  perception 
according  to  avatar’s  gender. 

Method/Tools 

For  accomplishing  these  goals,  we  generated 
a  male  and  a  female  virtual  avatar  with  five 
levels  of  strength  of  happy  and  anger  emo¬ 
tion  using  morphing  technique,  recruited  16 
healthy  and  normal  subjects  and  measuring 
the  subject’s  affected  emotion  by  rating  affec¬ 
tive  arousal  and  valence.  The  presentation  of 
expressions  of  virtual  faces  was  not  just  to 
show  a  still  image  but  to  show  an  animation 
in  order  for  the  subject  to  recognize  the  ava¬ 
tar’s  facial  expression  more  realistically. 
Therefore,  we  made  a  virtual  avatar  animate 
so  that  subjects  could  see  a  facial  animation 
changing  from  neutral  into  a  target  expres¬ 
sion  for  one  second,  subsequently  they  could 
see  the  target  expression  (one  of  five  levels) 
for  5  seconds.  Then,  they  scored  the  emo- 
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tional  valence  or  arousal  for  the  stimulus  while 
being  shown  a  black  screen  for  5  seconds. 

The  24  facial  expressions  of  avatars  were 
provided  in  random  order. 

Results 

In  this  study,  we  were  able  to  investigate  the 
human’s  perceptual  characteristics  evoked  by 
a  male  and  a  female  avatar’s  facial  expres¬ 
sion  of  happy  and  anger,  and  avatar’s  gradual 
facial  expression.  In  addition,  we  were  able  to 
know  that  the  virtual  avatar’s  facial  expression 
could  affect  human’s  emotion  and  it  showed 
different  characteristics  varied  with  avatar’s 
gender  and  the  strengths  of  the  facial  expres¬ 
sions  as  well  as  those  characteristics  ob¬ 
served  in  previous  studies  using  photos  or 
video  clips.  However,  we  could  also  see  that 
virtual  face  have  some  limitations  because  it 
is  not  real,  so  subjects  were  not  influenced  as 
much  as  they  recognized  the  virtual  faces. 

Conclusion 

This  study  was  meaningful  in  that  it  could  pro¬ 
vide  new  potential  for  using  or  manipulating 
emotional  intensity  by  controlling  a  virtual  ava¬ 
tar’s  facial  expression  linearly  using  a  morph¬ 
ing  technique,  although  a  virtual  avatar  has 
some  limitation  to  convey  its  emotion  using 
facial  expressions.  Therefore,  it  is  predicted  to 
be  used  for  assessing  emotional  characteris¬ 
tics  with  emotional  deficit  through  a  presenta¬ 
tion  of  dynamic  expression  having  various 
emotional  intensities. 

Contact: 

Jeonghun  Ku 

133-605  Dept,  of  Biomedical  Engineering 
Hanyang  University, 

Sung  Dong  P.O.  Box  55 
Seoul,  Korea, 

Tel :  +82-2-2290-0693 

Fax :  +82-2-2296-5943 

Email:  kuih@bme.hanvanq.ac.kr 


Presenter;  Jeonghun  Ku  PhD 

EMOTIONAL  INVOLVEMENT  IN  PRES¬ 
ENCE  DURING  INTERACTING  WITH 
VIRTUAL  AVATAR 

Jeonghun  Ku  Ph.D.,  Won  Yong  Hahn,  M.S., 
Hee  Jeong  Jang,  B.  A.,  Sung  Hyouk  Park, 
M.D.,  So  Young  Kim,  B.A.,  Kwang  Uk  Kim, 
M.S.,  Jae  Hun  Kim,  M.S.,  Jang  Han  Lee, 


Ph.D.,  Chan-Hyung  Kim,  M.D.,  Ph.D.,  Jae  Jin 
Kim,  M.D.,  Ph.D.,  In  Y.  Kim,  M.D.,  Ph.D.  and 
Sun  I.  Kim,  Ph.D. 

Dept,  of  Biomedical  Engineering,  Hanyang  Univer¬ 
sity,  Seoul,  Korea 

Background 

Technological  advances  make  virtual  reality 
(VR)  able  to  provide  social  events  by  being 
populated  with  avatars.  Several  studies  rep¬ 
resented  that  the  avatar’s  representation 
could  influence  people’s  sense  of  social  real¬ 
ism.  As  one  of  the  critical  components  to 
evaluate  the  effectiveness  of  virtual  environ¬ 
ment  for  the  social  interaction,  the  concept  of 
presence  has  been  used.  However,  there  are 
few  studies  on  how  emotion  works  in  the 
‘feeling  of  presence’  during  experiences  with 
virtual  societies,  although  emotion  plays  an 
important  role  in  social  interaction. 

Objectives 

It  focuses  on  the  impact  of  avatar’s  emotions 
(Happy,  Angry,  and  Neutral  Emotion)  in  co¬ 
presence,  social  presence,  and  tele¬ 
presence.  Specifically,  we  hypothesize  that 
presence,  particularly  tele-presence,  will  oc¬ 
cur  within  immersive  virtual  environments  as 
a  function  of  intensity  of  emotion.  Therefore, 
we  tried  to  evaluate  the  presence  produced 
by  emotional  value  of  virtual  avatars  in  virtual 
environments  in  this  study. 

Media/Tools 

A  total  of  fifteen  healthy  subjects  were  re¬ 
cruited  for  this  study.  2*3  (avatar  gender, 
avatar  emotion)  mixed  ANOVA  experimental 
design  was  used  for  this  experiment.  All  six 
avatars  have  emotional  appearance,  behav¬ 
iour,  and  voice.  Each  participant  experienced 
all  6  tasks  (avatar  gender  x  avatar  emotion) 
in  a  random  order.  Each  task  was  to  be  intro¬ 
duced  with  an  avatar  and  then  to  introduce 
oneself  to  the  avatar.  After  participating,  sub¬ 
jects  should  complete  a  presence  question¬ 
naire  and  answer  the  arousal  (relax:  0  -  exci¬ 
tation:  8)  and  valence  (unpleasant:  -4  - 
pleasant:  +4)  induced  by  the  experience. 

Results 

In  an  investigation  of  self-reported  co¬ 
presence  and  perceived  other’s  copresence, 
the  scores  of  both  co-presence  scales  to 
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happy  emotional  avatar  were  highest,  and  the 
scores  to  neutral  emotional  avatar  were  next, 
and  the  scores  to  angry  emotion  avatar  was 
lowest  (main  effect  :  p<0.001).  For  social 
presence  score,  the  significant  main  effect 
was  also  observed  (p=0.003).  And,  the  score 
to  happy  emotional  avatar  was  significantly 
high  to  Neutral  (p=0.001),  and  to  Angry  avatar 
(p=0.007).  The  scores  of  tele-presence  to 
happy  emotional  avatar  (p=0.014)  and  angry 
emotional  avatar  (p=0.046)  were  more  signifi¬ 
cantly  higher  than  the  score  to  neutral  emo¬ 
tion  avatar. 

In  the  results  of  correlation  analysis  between 
emotional  value  (arousal  and  valence)  and 
presence  score,  the  correlations  between  va¬ 
lence  and  self-reported  co-presence  (r=0.873, 
p<0.001)  and  perceived  others  co-presence 
(r=0.887,  p<0.001)  were  significant,  and  the 
correlation  between  tele-presence  and  abso¬ 
lute  value  of  valence  (r=0.327,  p=0.002)  and 
value  of  arousal  (r=0.228,  p=0.031)  was  also 
statically  significant.  Also,  the  correlation  be¬ 
tween  social  presence  and  absolute  value  of 
valence  was  statically  significant  (r=0.210, 
p=0.047). 

Discussion 

The  pattern  of  self-reported  copresence  and 
perceived  other’s  copresence  to  the  avatar’s 
emotional  expression  was  similar.  It  is  proba¬ 
bly  due  to  the  fact  that  the  questionnaires 
mostly  include  factors  regarding  intimacy  and 
involvement,  which  are  highly  related  to  emo¬ 
tional  valence.  On  the  other  side,  the  social 
presence  and  tele-presence  score  showed  a 
high  correlation  with  absolute  value  of  va¬ 
lence.  It  might  mean  that  humans  feel  greater 
presence  when  the  other  expresses  emotion 
regardless  of  positive  and  negative.  There¬ 
fore,  we  could  conclude  that  emotion  is  an 
important  factor  to  evoke  ‘feeling  presence’  in 
virtual  environments  with  avatars. 
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Background 

Virtual  reality  (VR)  has  become  a  representa¬ 
tive  method  or  tool  to  intervene  between  hu¬ 
man  and  computer.  It  has  been  widely  used 
in  various  fields  including  medical  field,  par¬ 
ticularly  for  treating  people  with  mental  dis¬ 
ease  and  training  handicapped  people.  It  is 
because  VR  has  the  ability  to  provide  us  a 
well  controlled  environment  as  well  as  a  real¬ 
istic,  interactive,  and  immersive  environment. 
Nowadays,  an  idea  has  emerged,  which  so¬ 
cial  stimuli  also  could  be  generated  using  VR 
technique,  particularly  using  avatars.  This 
idea  has  been  realized  and  utilized  for  peo¬ 
ple  with  social  anxiety.  However,  there  is  lack 
of  evidence  whether  a  virtual  avatar  could 
influence  human  behaviour  and  emotion. 
Therefore,  the  goal  of  this  study  was  to  de¬ 
velop  a  virtual  avatar  and  to  investigate  hu¬ 
man’s  behavioural  and  emotional  response  in 
order  to  examine  the  possibility  to  apply  it  to 
patients  with  schizophrenia. 

Method/Tools 

A  male  and  a  female  virtual  avatar  having 
emotional  expressions  (happy,  neutral,  or 
anger)  were  designed.  The  arousal  &  va¬ 
lence  of  each  avatar’s  facial  expression  was 
matched  to  the  same  level  based  on  results 
of  a  previous  study.  For  the  experiment,  2  x 
3  (avatar’s  gender  and  avatar’s  emotion) 
ANOVA  experimental  design  was  used.  For 
the  experiment,  fifteen  healthy  subjects  were 
recruited.  Each  participant  experienced  all  6 
introduction  tasks  (avatar  gender  x  avatar 
emotion)  in  random  order.  As  measurements 
for  this  study,  behavioural  response 
(interpersonal  distance  defined  as  a  distance 
between  a  subject  and  an  avatar),  emotional 
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(arousal  and  valence)  measurements,  and 
recalled  memory  score  were  acquired. 

Results 

Through  this  study,  we  were  able  to  observe 
that  an  avatar  expressing  emotion  could  influ¬ 
ence  human  behaviour,  emotion,  and  mem¬ 
ory.  In  detail,  when  the  interpersonal  distance 
was  compared  in  each  condition,  the  signifi¬ 
cant  main  effect  to  avatar’s  gender  was  ob¬ 
served  (p=0.023),  while  the  difference  accord¬ 
ing  to  avatar’s  gender  at  each  emotionai  ex¬ 
pression  was  not  significant.  In  addition,  fur¬ 
ther  distance  was  set  in  case  of  being  pre¬ 
sented  with  an  angry  avatar  regardless  of 
avatar’s  gender  (happy  vs  angry  :  p=0.006/ 
neutral  vs  angry  :  p=0.003).  In  summary,  hu¬ 
mans  keep  the  interpersonal  distance  closer 
to  female  avatar  than  to  male  avatars  and 
further  from  an  avatar  representing  angry 
emotion  than  to  an  avatar  representing  neu- 
trai  or  happy  emotion.  Furthermore,  subjects 
recalled  more  correctly  in  a  condition  having 
an  angry  avatar  than  in  a  neutral  avatar  condi¬ 
tion  (p=0.006).  However,  in  the  case  of  being 
provided  an  avatar  representing  happiness,  a 
higher  mean  score  was  shown  but  was  not 
significant  (p=0.108). 

Conclusion 

These  results  support  that  the  virtual  avatar 
could  influence  human  behaviour,  emotion, 
and  memory,  in  particular,  that  those  charac¬ 
teristics  were  varied  by  emotional  expressions 
of  the  virtual  avatar.  The  characteristics  re¬ 
sulting  from  this  study  coincided  with  studies 
conducted  with  real  people  or  other  instru¬ 
ments.  A  virtual  reality  and  virtual  avatar  could 
draw  human  behavioural  and  emotional  char¬ 
acteristics  and  assess  those  objectively. 
Therefore,  it  could  provide  a  potential  to  be 
used  for  assessing  human’s  characteristics  as 
well  as  assessing  and  enhancing  social  ability 
of  a  person  who  has  social  function  deficits, 
e.g.,  schizophrenia,  using  virtual  avatars. 
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In  this  project  we  are  developing  an  effective 
feedback  system  for  a  human  interface  with  a 
video  game  designed  to  promote  mental  and 
physical  relaxation.  Therapies  such  as  Tai 
Chi  and  Yoga  have  been  shown  to  have 
many  therapeutic  benefits  in  rehabilitation 
including  enhancing  postural  awareness  and 
reducing  chronic  pain  and  hypertension. 

We  aim  to  develop  a  computer  game  in 
which  the  player's  performance  is  determined 
by  their  ability  to  master  one  of  these  physi¬ 
cal  arts  and  in  doing  so  bring  about  a  state  of 
mental  relaxation.  We  are  developing  a  vir¬ 
tual  reality  application  where  whole-body 
movement,  respiration  and  physiologicai  cor¬ 
relates  of  relaxation  are  used  as  input.  Such 
input  will  be  processed  to  provide  changes  to 
the  avatar  or  its  environment  in  order  to  af¬ 
fect  appropriate  biofeedback  for  the 
user.  The  purpose  of  this  biofeedback  is 
such  that  they  may  more  easily  achieve  tar¬ 
geted  postures  and  physiological  variable 
ranges  commensurate  with  Yoga/Tai  Chi 
philosophy. 

A  garment  based  physiological  and  kinemat¬ 
ics  measurement  system  wiil  provide  our  pri¬ 
mary  method  of  communication  with  the 
video  game.  This  lightweight  garment  will 
monitor  heart  and  breathing  rate,  muscle  ac¬ 
tivity,  galvanic  skin  response,  and  alignment 
of  body  segments.  To  measure  the  human 
body  motion  we  have  designed  and  imple¬ 
mented  a  Universal  Serial  Bus  (USB)  based 
unobtrusive  kinematics  transducer  capable  of 
scalable  deployment  with  minimum  instru¬ 
mentation. 

The  player  must  reproduce  physical  postures 
displayed  by  a  model  on  screen  whilst  main¬ 
taining  a  relaxed  and  controlled  breathing 
pattern.  Feedback  is  provided  by  means  of 
reproduction  of  the  player's  body  image  on 
screen.  Quality  of  movement  and  posture. 
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and  degree  of  mental  relaxation  will  determine 
the  player's  performance.  The  game  could 
take  the  form  of  a  track  and  field  decathlon 
format.  This  system  could  benefit  patients 
suffering  from  many  conditions  including  hy¬ 
pertension,  anxiety  disorder,  and  chronic  pain. 
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Background 

Applications  of  Virtual  reality  (VR)  as  a  dis¬ 
traction  technique  have  included  situations 
requiring  medical  procedure  for  an  injury/ 
ailment.  Common  invasive  procedures 
(injections/venipuncture)  involve  the  learner 
practicing  the  task.  Some  health  training  pro¬ 
grams  offer  students  the  opportunity  to  learn 
invasive  techniques  on  each  other  prior  to 
performing  them  in  therapeutic  situations.  The 
aim  of  this  study  was  to  determine  if  VR  was 
effective  in  reducing  pain  and  anxiety  in  stu¬ 
dents  receiving  a  digital  nerve  block  from  their 
peers  as  part  of  a  teaching  program. 

Method/tools 


A  randomized  controlled  trial  was  used.  Po¬ 
diatry  undergraduates  were  randomly  as¬ 
signed  to  receive  the  procedure  whilst  im¬ 
mersed  in  VR  (Group  A=VR)  or  verbally  dis¬ 
tracted  (Group  B=control).  Pre  and  post  out¬ 
come  measures  included  VAS(pain),  State- 
Trait  anxiety  inventory  (STAI),  and  simulator 
sickness  questionnaire  (SSQ).  Blood  pres¬ 
sure  (BP),  heart  rate  (HR),  skin  conductance 
(SC),  and  malaise  scale  (MS)  were  meas¬ 
ured  pre,  during,  and  post  procedure.  Wthin- 
group  and  between-group  analyses  were 
performed. 

Results 

There  were  no  significant  differences  in 
demographics  between  groups.  No  signifi¬ 
cant  difference  was  found  between  groups 
for  VAS  for  pain  (p=0.78).  Compared  to 
baseline  levels,  SC  was  significantly  higher 
during  the  procedure  in  the  control  group 
(p<0.001).  State  STAI  significantly  decreased 
in  both  groups  following  the  procedure 
(Group  A  p=0.01 ;  Group  B  p=0.008)  and  no 
difference  was  evident  between  groups  (pre 
(p=0.124),  post  (p=0.065)  procedure).  Two 
subjects  reported  symptoms  (MS)  when  im¬ 
mersed  in  VR.  There  was  no  significant  dif¬ 
ference  between  pre/post  SSQ  in  the  VR 
group.  Diastolic  BP  was  significantly  lower 
(p=0.034)  and  HR  significantly  higher 
{p=0.042)  in  the  VR  group.  Novelty/  discus¬ 
sion:  To  date  the  majority  of  studies  investi¬ 
gating  the  use  of  VR  as  a  distraction  tech¬ 
nique  have  concentrated  on  medical  proce¬ 
dures  with  high  pain  levels.  Pain  associated 
with  injections  is  comparatively  low,  however 
often  associated  with  high  anxiety  levels.  No 
significant  difference  in  subjective  pain  may 
be  a  result  of  the  action  of  local  anaesthetic, 
numbing  the  area  after  needle  insertion.  The 
significant  increase  in  SC  during  the  control 
condition  only  (p<0.001)  suggests  that,  de¬ 
spite  the  presence  of  local  anaesthetic,  an¬ 
ticipatory  anxiety  was  high  when  the  proce¬ 
dure  was  viewed,  compared  to  distraction 
with  VR  (p>0.056).  STAI  results  are  possibly 
due  to  an  ordering  effect;  the  majority  of 
group  A  subjects  received  the  procedure 
then  performed  the  procedure  on  another 
student.  Malaise  scale  results  indicate  that 
adverse  effects  occurred  in  two  subjects.  No 
significant  change  in  SSQ  scores  for  the  VR 
group  supports  MS  findings.  However,  pre 
and  post  SSQ  scores  were  high  for  both 
groups,  including  the  control  group  where 
SSQ  scores  should  be  zero  since  subjects 
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were  not  exposed  to  VR,  suggesting  that  SSQ 
may  be  associated  with  procedural  anxiety 
and  not  adverse  VR  symptoms.  Significant 
BP  and  HR  changes  during  VR  reflect  the 
physical  activity  involved  in  the  intervention. 

Conclusion:  While  VR  did  not  significantly 
influence  subjective  pain  scores,  lower  SC 
indicates  that  VR  reduced  anticipatory  anxiety 
associated  with  the  procedure.  The  MS  may 
be  a  more  appropriate  measure  of  adverse 
effects  than  the  SSQ  in  situations  where  high 
levels  of  anxiety  exist. 
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Background 

The  Virtual  reality  (VR)  ARQuake  game  was 
developed  at  the  Wearable  Computer  Labo¬ 
ratory  (UniSA)  and  is  currently  being  used  by 
researchers  as  a  distraction  technique  for 
children  undergoing  painful  procedures.  This 
study  was  designed  to  understand  how  chil¬ 
dren  react  to  VR,  when  not  undergoing  pain¬ 
ful  situations.  The  aim  of  the  study  was  to 
compare  the  subjective,  physiological  and 
adverse  responses  of  children  to  the  VR 
ARQuake  game  with  an  animated  movie. 

Method/tools 

A  randomized  AB/BA  crossover  design  was 
used.  Subjects  aged  between  6-17  years 
were  randomly  assigned  to  14  minutes  of  VR 
and  a  14  minute  section  of  an  animated 
movie  in  varying  order  (Group  A=VR/movie; 
group  B=movieA/R).  The  simulator  sickness 
questionnaire  (SSQ)  was  completed  pre  and 
post  procedure.  Blood  pressure  (BP),  heart 
rate  (HR),  skin  conductance  (SC),  and  mal¬ 
aise  scale  (MS)  were  measured  pre,  during, 
and  post  procedure.  Subjects  completed  a 
presence  questionnaire  (PQ)  following  both 
VR  and  movie. 

Results 

Thirty-six  subjects  were  recruited  (21  males 
and  15  females,  mean  age  11.89±2.98).  Sub- 
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ject  numbers  were  equal  in  both  groups.  No 
significant  difference  was  found  in  demo¬ 
graphic  details  between  groups.  Significant 
treatment  effects  were  present  for  the  level  of 
enjoyment  of  the  game  (PQ)  (p=0.03)  and 
SSQ  (p<0.0001).  No  significant  period-by¬ 
treatment  interaction  was  present  for  any  out¬ 
come  measures  (p>0.05).  The  level  of  pres¬ 
ence  (PQ)  was  significantly  higher  during  VR 
(p<0.004).  No  significant  difference  in  SC  be¬ 
tween  VR  and  movie  (>0.051)  occurred  at  any 
time  point.  Blood  pressure  and  HR  were  sig¬ 
nificantly  higher  during  VR  (p<0.014),  how¬ 
ever  no  significant  difference  in  BP  and  HR 
was  present  post  5  minutes  (p>0.481).  Symp¬ 
toms  (e.g.,  HMD  heavy,  dizziness)  were  re¬ 
ported  during  VR  on  the  MS  by  47%  of  sub¬ 
jects,  compared  to  3%  during  the  movie.  Mild 
nausea  was  reported  by  6%  of  subjects  dur¬ 
ing  VR  (0%  movie).  Subjects  reported  no 
symptoms  5  minutes  post  VR. 

Novelty/  discussion 

The  VR  ARQuake  game  demonstrated  higher 
levels  of  presence  than  the  movie  with  sub¬ 
jects  reporting  significantly  higher  scores  in 
the  PQ  (felt  like  they  were  really  there;  felt  like 
they  were  actually  moving;  game  was  more 
difficult  and  harder  to  get  used  to).  These  re¬ 
sults  indicate  that  VR  will  be  an  effective  dis¬ 
traction 

technique  during  painful  procedures  because 
of  the  higher  level  of  presence  and  greater 
level  of  concentration  required.  Although 
greater  symptoms  were  reported  on  the  MS 
during  VR,  only  6%  reported  mild  nausea  and 
all  symptoms  subsided  within  five  minutes.  It 
appears  that  the  MS  is  a  more  appropriate 
indicator  of  adverse  responses  to  VR  than 
SSQ  in  children,  possibly  as  a  result  of  the 
wording  on  the  SSQ.  The  non-significant  SC 
result  between  VR  and  movie  supports  the 
MS  findings.  The  significant  increase  in  BP 
and  HR  during  VR  reflect  the  physical  activity 
involved  in  interacting  with  the  virtual  environ¬ 
ment. 

Conclusion 

This  study  provides  more  information  about 
the  physiological  and  adverse  effects  of  VR  in 
children  and  provides  support  for  the  use  of 
the  VR  ARQuake  game  in  children  undergo¬ 
ing  painful  and  anxiety  provoking  procedures 
provided  a  malaise  scale  is  used  to  monitor 
nausea. 
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The  aim  of  this  study  was  to  determine  if  Vir¬ 
tual  Reality  is  an  effective  distraction  tech¬ 
nique  for  children  undergoing  minor  medical 
procedures  in  a  Paediatric  Emergency  De¬ 
partment  compared  to  watching  a  movie. 
Children  between  6-17  years  undergoing 
minor  medical  procedures  such  as  blood 
sampling,  intravenous  cannulae  insertion  and 
suturing  were  provided  with  either  Virtual 
reality  or  an  animated  movie  excerpt  as  a 
distraction  technique  during  the  procedure. 
Pre,  post  and  maximum  scores  were  re¬ 
ceived  for  children's  self  report  pain  and  anxi¬ 
ety  and  parent's  perception  of  their  child's 
pain  and  anxiety.  Staff  members  were  asked 
to  provide  a  score  for  their  perception  of  the 
child's  pain  and  anxiety.  Parental  anxiety  was 
also  measured  for  comparison  with  the 
child's  anxiety  levels.  The  brief  behavioral 
distress  scale  was  used  to  score  the  child's 
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behavior  during  the  procedure.  Data  collection 
for  this  study  is  still  ongoing  and  is  anticipated 
to  be  completed  by  June  2005.  To  date, 
twenty-two  subjects  have  been  recruited  with 
a  target  of  at  least  100  subjects.  The  two  in¬ 
terventions  (Virtual  reality  and  animated 
movie)  will  be  compared  using  mixed  be- 
tween-within  subjects  analysis  of  variance 
and  independent  t  tests.  Baseline  pain  and 
anxiety  data  for  these  procedures  was  col¬ 
lected  in  a  separate  study,  providing  informa¬ 
tion  about  the  pain  and  anxiety  associated 
with  these  procedures  without  distraction.  Pi¬ 
lot  testing  comparing  the  level  of  presence 
and  enjoyment  of  the  Virtual  reality  to  watch¬ 
ing  an  animated  movie  in  healthy  children  not 
undergoing  procedures  has  also  been  com¬ 
pleted.  If  Virtual  reality  is  found  to  significantly 
reduce  pain  and  anxiety  in  children  during 
minor  medical  procedures  compared  to 
watching  an  animated  movie,  the  technique 
may  be  employed  by  the  Pediatric  Emergency 
Department  as  a  non  pharmacological  pain 
management  tool. 
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Research  Status:  Completed 

Background  &  Significance  of  the  Prob¬ 
lem 

Virtual  environments  (VEs)  can  be  used  to 
assess  spatial  memory  and  navigation  skills, 
although  the  behaviour  elicited  is  inherently 
complex.  The  purpose  of  this  study  was  to 
investigate  the  ecological  validity  and  interre¬ 
lationship  between  virtual  navigation  tasks 
and  several  neuropsychological  tests. 

Methods/Toots 

Forty-two  participants  (mean  age:  70.42) 
took  part  in  the  study,  of  which  eighteen  were 
recruited  from  the  Cognitive  Neurology  Clinic 
at  Sunnybrook  &  Women’s  College  Health 
Sciences  Centre  (S&W)  based  on  diagnosis 
of  mild  cognitive  impairment  (MCI).  Subjects 
completed  the  Groton  Maze  Learning  Test 
(GMLT;  a  hidden  maze  task  developed  by 
one  of  the  authors  [P.J.S.])  and  several 
neuropsychological  tests,  including  the 
MMSE,’  the  Rey  Complex  Figure  Test,^  Ben¬ 
ton  Visual  Retention  test,®  Trail-Making  Test 
(Forms  A  and  B),'*  and  Digit  Span.®  Sixteen 
of  the  participants  (9  normal,  7  MCI)  also 
undertook  a  VE  navigation  task.®  Behaviour 
elicited  by  the  VE  task  was  characterized  in 
terms  of  two  components,  as  estimated  by 
factor  analysis:  a  VE  memory  index,  and  a 
VE  movement  index. 

Results 

The  VE  memory  index  was  significantly  asso¬ 
ciated  with  the  results  of  a  conventional 
memory  test  (Rey  Complex  Figure),  and  the 
VE  movement  index  was  significantly  associ¬ 
ated  with  measures  of  the  Trail  Making  Test. 
Compared  with  normal  participants,  partici¬ 
pants  with  mild  cognitive  impairment  (MCI) 
showed  a  significant  reduction  in  the  mem¬ 
ory-related  measures  in  the  PGRD  Maze 
Learning  Test  and  memory  tests  (particularly 
RCFT),  and  the  MCI  group  deviated  signifi¬ 
cantly  from  the  correct  route,  particularly  in 
novel  environments.  Performance  in  the  VE 
navigation-learning  task  was  significantly 
associated  with  measures  of  visual  memory 
and  the  executive  function  in  conventional 
neuropsychological  tests. 
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Conclusion 

The  results  of  this  study  demonstrate  that  VE 
technology  can  be  used  to  assess  spatial 
memory  and  navigation  skills,  and  that  meas¬ 
ures  of  this  behavior  in  a  VE  are  related  to 
those  of  conventional  neuropsychological 
tests.  Our  results  also  demonstrate  that  the 
MCI  group  showed  a  significant  reduction  in 
memory-related  measures  in  the  GMLT  and 
the  neuropsychological  tests.  In  particular, 
differences  between  the  groups  were  evident 
in  the  immediate  and  delayed  recall  in  the 
Rey  Test.  However,  because  of  the  small 
sample  size,  statistically  significant  results 
were  not  found  between  groups  in  the  VE 
navigation  task. 

Novelty 

The  interrelationship  between  the  VE  task,  the 
GMLT,  and  neuropsychological  assessments 
is  important  for  administration  and  interpreta¬ 
tion  of  these  computerized  tests  in  the  future. 
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Research  status:  Still  in  progress 

Children  with  autism,  as  well  as  children  with 
other  developmental  disabilities,  may  have 
dysfunctional  sensory  systems.  Sometimes 
one  or  more  senses  are  either  over-  or  un¬ 
der-reactive  to  stimulation  in  these  individu¬ 
als.  Sensory  processing  abnormalities  affect 
all  aspects  of  adaptive,  cognitive,  social,  and 
academic  functioning,  and  correlate  with 
higher  levels  of  stereotypic,  rigid,  and  repeti¬ 
tive  behaviours  in  autism.  Thus,  sensory 
processing  abnormalities  may  be  important 
to  address  in  therapeutic  interventions  that 
aim  to  reduce  rigidity  and  stereotyped  behav¬ 
iours.  Sensory  integration  therapy  is  based 
on  a  theory  developed  by  Ayres  (1972), 
which  emphasizes  the  relationship  between 
sensory  experiences  and  motor  and  behav¬ 
ioural  performance.  Intervention  strategies 
involve  the  use  of  planned  and  controlled 
sensory  experiences,  including  vestibular, 
proprioceptive,  and  somatosensory  activities, 
such  as  swinging,  deep  pressure  touch,  and 
tactile  stimulation;  that  is,  the  techniques  fa¬ 
cilitate  attention  and  awareness,  and  reduce 
overall  arousal  of  children  with  autism. 

Virtual  reality  technology  is  an  exciting  tool 
for  allowing  children  with  autism  to  practice 
behaviours  in  role-play  situations,  while  pro¬ 
viding  a  safe  environment  for  rule  learning 
and  repetition  of  tasks  (Parsons  &  Mitchell, 
2002).  However,  some  ethical  and  technical 
concerns  surround  the  use  of  fully  immersive 
virtual  reality  technology  (i.e.,  the  use  of 
head-mounted  displays  (HMDs)).  HMDs  can 
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be  extremely  expensive  and  people  may  ex¬ 
perience  ‘cybersickness’  in  the  form  of  nau¬ 
sea,  headache,  and  dizziness,  whereas  desk¬ 
top  or  projection  virtual  environments  tend  to 
be  much  less  likely  to  induce  cybersickness. 
Moreover,  because  HMDs  place  some  limita¬ 
tions  on  the  child’s  interaction  with  another 
person,  the  mixed  and  augmented  reality  is 
more  useful  for  group  interactions  and  sen¬ 
sory  experiences. 

Therefore,  we  developed  a  virtual  reality- 
tangible  (VR-tangible)  interaction  system  for 
sensory  integration  training  and  assessment 
of  autism  by  mixing  the  virtual  environment 
with  the  natural  physical  environment.  With 
this  system,  children  with  autism  experience 
vestibular,  proprioceptive,  and  somatosensory 
activities,  such  as  swinging,  spinning,  and 
rotating,  and  a  social  skills  training  program 
such  as  eye  gaze.  In  addition,  a  program  that 
involves  breaking  virtual  balloons  with  a  real 
stick  measures  the  visuomotor  coordination  of 
children  with  autism.  The  participants  view 
themselves,  virtual  balloons,  and  a  real  stick 
on  a  large  screen  that  displays  game-like  sce¬ 
narios.  Our  VR-tangible  interaction  system 
consists  of  a  Pentium  IV  PC,  a  projector,  a 
screen  (200  x  150  cm),  an  infrared  reflector, 
and  a  digital  camera. 

The  performances  of  normal  control  groups 
and  autistic  groups  will  be  compared  in  the 
visuomotor  coordination  program  and  the  so¬ 
cial  skills  training  program.  The  degree  that 
the  sensory  integration  training  leads  autistic 
children  to  more  productive  contacts  with  an¬ 
other  individual  and  environments  will  also  be 
analyzed. 

As  the  study  is  still  in  progress,  final  results 
were  not  available  at  the  time  this  abstract 
was  written.  The  use  of  a  VR-tangible  interac¬ 
tion  system  to  better  understand  sensory 
processing  abnormalities  of  children  with  au¬ 
tism  is  relatively  rare,  even  in  studies  with  vir¬ 
tual  environments.  We  hope  that  this  study 
will  clarify  sensory  processing  abnormalities 
of  children  with  autism  and  stimulate  further 
studies  on  sensory  integration  training  and 
assessment  in  the  field  of  VR-tangible  inter¬ 
action. 
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SEX  DIFFERENCES  IN  HEALTHY  ELD¬ 
ERLY  ADULTS  FOR  COMPLETING  2- 
DIMENSIONAL  HIDDEN  MAZE  AND  3- 
DIMENSIONAL  VIRTUAL  REALITY 
NAVIGATION  LEARNING  TASKS 
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Background  &  Significance  of  the  Probiem 

Sex  differences  in  navigation  and  maze 
learning  are  well  known,  with  men  often  out¬ 
performing  women  on  route-learning,  spatial 
perception,  and  visualization. Some  sug¬ 
gest  that  these  differences  stem  from  evolu¬ 
tion,  while  others  attempt  to  further  elucidate 
these  differences  by  new  experimental  meth¬ 
ods.  The  present  study  used  a  series  of  non¬ 
verbal,  visually-guided  navigation  tasks  to 
investigate  gender  differences  in  healthy, 
elderly  adults. 

Methods/Tools 

The  three  distinct  test  components  were  1) 
the  Groton  Maze  Learning  Test  (GMLT),  2) 
traditional  neuropsychological  tests,  and  3)  a 
3D  VE  navigation-learning  task.  The  GMLT  is 
a  computerized  measure  of  a  participant’s 
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ability  to  acquire  and  use  efficiently  an  internal 
spatial  map  of  a  hidden  2D  maze.  Neuropsy¬ 
chological  tests  included  the  MMSE,®  the  Rey 
Complex  Figure  Test,®  Benton  Visual  Reten¬ 
tion  test,^  Trail-Making  Test,®  and  Digit  Span.® 
The  3D  VE  navigation  task^°  involves  learning 
and  recalling  routes  through  a  virtual  city. 
Twenty-five  healthy  elderly  volunteers  partici¬ 
pated. 

Results 

Sex  differences  were  found  in  learning  ability 
with  the  2D  maze  test  and  the  VE  navigation. 
Men  excelled  at  maze-learning  ability,  espe¬ 
cially  after  the  middle  trials,  although  men  and 
women  did  not  differ  in  performance  on  the 
first  learning  trial  for  this  test.  These  results 
suggest  that  men  more  easily  acquired  spatial 
ability  and  learning  than  women,  and  that  men 
also  reached  their  peak  performance  ability 
more  rapidly  than  women  in  the  maze¬ 
learning  test. 

Men  significantly  outperformed  the  women 
only  on  the  first  trial  of  the  VE  navigation  task. 
There  were  no  sex  differences  after  a  second 
trial,  perhaps  due  to  the  relative  ease  of  the 
VE  navigation  task  since  no  subjects  deviated 
significantly  from  the  path  following  the  sec¬ 
ond  learning  trial. 

Performance  on  the  GMLT  was  related  to  per¬ 
formance  on  both  the  memory-related  neuro¬ 
psychological  tests  and  the  VE  task. 

Conclusion 

Men  outperform  women  on  the  2D  and  3D 
navigation  tasks.  Future  studies  should  inves¬ 
tigate  sex  differences  between  topographic 
and  Euclidian  cues  in  VE  navigation  learning 
and  how  these  strategies  and  age  affect  spa¬ 
tial  memory  and  navigation  learning  in  normal 
people. 

Novelty 

This  study  suggests  that  the  VE  task  has  eco¬ 
logical  and  comparative  validity  for  measuring 
spatial  ability  and  navigation. 
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WHAT  ASPECTS  OF  VIRTUAL  ENVI¬ 
RONMENTS  MAKES  THE  EXPERI¬ 
ENCE  “REAL”  FOR  AGORAPHOBIC 
PATIENTS? 

David. G.Walshe\  Elizabeth  J.  Lewis^ 

'  St.  Stephen's  Hospital,  Cork,  Ireland.,  Depart¬ 
ment  of  Psychiatry,  University  College  Cork. 

^  St.  Stephen's  Hospital,  Cork,  Ireland.  Depart¬ 
ment  of  Child  Health  and  Paediatrics,  University 
College  Cork 

Research  Status:  Research  in  progress 

Agoraphobia  with  or  without  panic  disorder  is 
a  disabling  mental  health  problem  with  an 
estimated  population  prevalence  of  6%.  It  is 
frequently  seen  in  both  primary  and  secon¬ 
dary  mental  health  settings.  Currently,  al¬ 
though  the  evidence  base  for  behaviour  ther¬ 
apy  for  agoraphobia  is  extensive,  the  service 
delivery  is  poor.  It  is  estimated  that  25%  of 
phobic  patients  refuse  “in  vivo”  exposure 
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therapy  or  drop  out.  Consequently  behaviour 
therapy  is  often  difficult  to  provide  in  this  area. 

Preliminary  studies  have  shown  a  role  for  vir¬ 
tual  reality  (VR)  in  the  treatment  of  agorapho¬ 
bia.  Vincelli  et  al.  showed  significant  reduc¬ 
tions  in  the  number  of  panic  attacks,  level  of 
depression  and  both  state  and  trait  anxiety 
foliowing  VR  treatment  with  Experiental  Cog¬ 
nitive  Therapy  (ECT).^  Botella  et  al  showed 
VR  exposure  therapy  as  effective  as  in  vivo 
exposure  in  a  number  of  outcome  measures 
including  fear,  avoidance  and  panic  symp¬ 
toms.^ 

In  this  study  of  5  agoraphobic  patients  we  in¬ 
vestigated  what  aspects  of  virtual  environ¬ 
ments  induce  presence  in  agoraphobic  pa¬ 
tients  using  VRMRI  Virtual  World  software  by 
Psychology  Software  Tools,  Inc.  This  software 
provides  13  inter-connected  VR  environments 
which  include;  an  urban  area,  apartment, 
theatre  complex,  restaurant,  bank,  urban  sub¬ 
way  station,  village  subway  station,  village 
area,  house,  doctor’s  office,  airport  subway 
station,  and  two  airports). 

We  examined  which  environments  induced 
presence  with  associated  subjective  distress 
and  physiological  anxiety  and  whether  the 
addition  of  background  sounds  pertinent  to 
each  environment  influenced  presence  and 
distress 
levels. 

Procedure 

5  subjects  with  DSM-IV  criteria  either  for  ago¬ 
raphobia  without  history  of  panic  disorder  or 
panic  disorder  with  agoraphobia  were  initially 
exposed  to  a  neutral  VR  environment  to  accli¬ 
matise  to  VR.  Thereafter  patients  navigated 
virtual  environments  with  or  without  the  addi¬ 
tion  of  background  sound  over  a  3  minute  pe¬ 
riod  unless  anxiety  symptoms  necessitated 
termination  of  exposure. 

Ratings  of  presence  (Item  from  Presence  In¬ 
ventory),  subjective  distress  (SUDS)  and 
physiological  anxiety  (Heart  Rate  measure¬ 
ments)  were  recorded  once  per  minute  during 
exposure  to  each  environment  and  verbal 
feedback  from  each  patient  following  each 
exposure  was  recorded. 

Results 

This  study  is  ongoing.  Results  will  be  pre¬ 


sented  at  conference  together  with  VR  envi¬ 
ronments. 

Novelty 

VRET  for  agoraphobia  is  at  a  developmental 
stage.  Information  from  this  study  will  assist 
in  tailoring  VR  software  and  treatment  pro¬ 
grams. 
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IS  VIDEO  HOMEWORK  OF  BENEFIT 
WHEN  PATIENTS  DON’T  RESPOND 
TO  VR  THERAPY  FOR  DRIVING  PHO¬ 
BIA? 

David. G.Walshe\  Elizabeth  J.  Lewis^ 


’  St.  Stephen's  Hospital,  Cork,  Ireland.,  Depart¬ 
ment  of  Psychiatry,  University  College  Cork. 

^  St.  Stephen's  Hospital,  Cork,  Ireland.  Depart¬ 
ment  of  Child  Health  and  Paediatrics,  University 
College  Cork 

In  our  Virtual  Therapy  Clinic,  based  in  Cork, 
we  have  specialised  in  developing  and  evalu¬ 
ating  virtual  reality  environment  treatment 
(VRET)  programmes  for  driving  phobia  in 
motor  accident  victims.  Findings  have  been 
reported  supporting  this  treatment  modality.^' 
^  This  patient  group  often  has  co-morbid  psy¬ 
chiatric  conditions  including  major  depres¬ 
sion,  posttraumatic  stress  disorder,  panic 
disorder,  and  substance  abuse  which  compli¬ 
cates  treatment. 

Our  standard  driving  programme  involves  8- 
12  VR  sessions,  using  computer  games  with 
graded  driving  in  vivo  homework  tasks  follow¬ 
ing  progress  in  virtual  exposure. 

Presently  we  are  focusing  on  two  clinical 
concerns.  Patients  who  do  not  desensitise/ 
reprocess  their  travel  fears  despite  multiple 
virtual  exposure  sessions  and  the  number  of 
treatment  sessions  required  for  successful 
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VR  treatment. 

Aim 

To  assess  whether  the  addition  of  video 
homework  tasks  involving  exposure  to  watch¬ 
ing  video  clips  of  driving  situations  encoun¬ 
tered  in  VR  therapy  will  bring  about  desensi¬ 
tization  and  thereby  symptom  resolution  in 
patients  resistant  to  VR  exposure  therapy. 

Procedure 

3  patients,  with  a  diagnosis  of  DSM-IV  Spe¬ 
cific  Phobia-driving,  resistant  to  a  VR  driving 
programme  undertook  a  “video  homework” 
programme.  A  video  tape  containing  a  10 
minute  clip  of  anxiety  provoking  driving  situa¬ 
tions  derived  from  the  most  recent  VR  session 
was  assigned  as  homework.  These  clips  were 
recorded  directly  from  the  VR  driving  scenes 
presented  in  therapy.  Each  patient  was  re¬ 
quested  to  watch  the  video  clip  for  30  minutes 
daily  in  a  darkened  room  with  earphones  and 
record  Subjective  Units  of  Distress  (SUD)  rat¬ 
ings  daily  until  anxiety  level  fell  to  SUD  rating 
of  2  or  less.  As  the  patient  progressed  to 
more  distressing  driving  scenarios  in  VR  ther¬ 
apy  the  video  clip  was  updated  accordingly. 

The  Fear  of  Driving  Inventory  (FDI)  and  the 
Posttraumatic  Stress  Disorder  Symptom 
Scale  (PTSS)  were  documented  at  entry  and 
reassessed  on  completion  of  the  study. 

Results 

This  study  is  ongoing.  Results  will  be  pre¬ 
sented  at  conference  together  with  assigned 
video  clips. 

Novelty 

This  is  a  novel  development  in  VRET  with 
potential  clinic  benefit. 
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A  PRIMER  ON  TATRC’S  MEDICAL 
MODELING  &  SIMULATION  PORTFO¬ 
LIO:  THE  SERIOUS  SIDE  OF  FUN 

J.  Harvey  Magee 

Portfolio  Manager,  Medical  Modeling  &  Simulation 
(MM&S) 

Telemedicine  and  Advanced  Technology  Re¬ 
search  Center  (TATRC) 

HQ  U.S.  Army  Medical  Research  and  Materiel 
Command 

As  we  enter  the  21®'  century,  the  worlds  of 
military  and  civilian  medical  training  struggle 
with  critical  questions.  For  the  military,  how 
do  we  train  medical  personnel  in  peace  for 
the  realities  of  war?  For  the  civilian  sector, 
how  do  we  train  medical  personnel  in  a  way 
that  results  in  increased  patient  safety... or 
reduced  patient  errors?  Reports  from  the 
U.S.  General  Accounting  Office  and  the  Insti¬ 
tute  of  Medicine  have  dealt  with  these  chal¬ 
lenges. 

Why  serious?  Here  are  a  couple  of  exam¬ 
ples.  First,  health  care  the  power  to  heal  or 
harm  patients.  How  we  train  to  deliver  care 
is  serious  too.  What  if  residents  actually  de¬ 
liver  health  care  the  way  we  teach  them  to? 
Second,  the  human  body  is  infinitely  chal¬ 
lenging...  and  constantly  changing.  Every¬ 
body  (literally  every  body)  is  different.  How 
much  realism  is  necessary  to  simulate  it? 
What  objective  performance  metrics  are  most 
appropriate?  Simulated  environments  impact 
people,  but  in  what  ways? 
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In  2000,  TATRC  MM&S  leaders  (Dr.  Moses, 
Mr.  Magee),  facilitated  a  70-person  strategic 
planning  workshop.  At  its  conclusion  and  in 
conjunction  with  “end  users”  of  simulation- 
based  training  systems,  a  strategy  was  devel¬ 
oped.  Based  on  that  strategy,  collaborations 
of  academic,  government,  and  industrial  re¬ 
search  groups  began  to  take  shape,  and  net¬ 
works  of  relationships  formed.  Leaders 
stepped  forward,  and  funding  sources  were 
identified  to  propel  research  in  four  general 
categories  of  medical  simulation  research; 
PC-based  interactive  multimedia,  digitally  en¬ 
hanced  mannequins,  part-task  trainers,  and 
Total  Immersion  Virtual  Reality.  Key  defining 
events  and  workshops  have  been  funded  over 
the  past  five  years  in  support  of  that  strategy. 
Annual  refinements  of  the  strategy  have  been 
made,  always  in  conjunction  with  “end  users.” 

Fueled  by  congressionally  designated  funded 
and  managed  by  TATRC,  the  Center  for  Inte¬ 
gration  of  Medicine  and  Innovative  Technol¬ 
ogy  (CIMIT)  Simulation  Group  (PI:  Dr.  Steve 
Dawson)  embraced  the  challenge  to  develop 
and  integrate  key  enabling  technologies  into 
systems  of  simulation-based  training.  Exam¬ 
ples  are  real-time  in  vivo  tissue  property 
measurement,  tissue-tool  interactions,  graph¬ 
ics  and  visualization,  learning  systems,  met¬ 
rics  development,  learning  transfer  and  as¬ 
sessment. 

Examples  of  current  projects  include  ad¬ 
vanced  ureteroscopic  surgical  simulation 
(TURP  procedure),  virtual  cricothyroidotomy, 
VIRGIL™  Chest  Trauma  Training  System, 
Central  Venous  Catheterization,  Exsanguinat¬ 
ing  Hemorrhage,  “VR-Demo”,  Dynamic  Injury 
Creation  Simulator. 

What  degree  of  validation  is  necessary  to 
demonstrate  transfer-of-training  effective¬ 
ness?  What  open  source  standards  require 
development  enroute  to  the  eventual  need  for 
systems  to  be  interoperable  with  either  other 
medical  simulation  systems  or  more  compre¬ 
hensive  training  systems? 

In  summary,  the  presenter  will  predict  some 
benefits  of  medial  simulation,  and  offer  some 
closing  thoughts  for  success  for  the  MM&S 
community... and  the  world. 


Presenter:  Ralph  Mager  MD 
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MEASUREMENTS  IN  A  REAL-CAR 
BASED  DRIVING-SIMULATOR 

Dr.  Ralph  Mager 

COAT  Basel,  Switzerland 

Objective 

The  goal  of  the  present  study  was  to  evalu¬ 
ate  physiological  measures  and  objective 
performance  parameters  during  driving  in  a 
real-car  based  driving  simulator.  Arousing 
auditory  stimuli  were  applied  to  compare  data 
prior  and  after  intervention  to  test  the  sensi¬ 
tivity  of  the  system. 

Design  and  Methods 

Overall  41  subjects  were  selected  matched 
for  age  and  driving  experience.  To  create 
realistic  traffic  scenarios  in  a  laboratory  envi¬ 
ronment  a  passenger  car  simulator  was  used 
emulating  the  functionality  of  a  modern  car. 
Electroencephalographic  (EEG)  activity,  skin 
conductance,  respiratory  and  cardiac  pa¬ 
rameters  were  continuously  recorded  during 
driving.  Analysis  was  focused  on  time  inter¬ 
vals  prior  and  after  application  of  a  warning 
stimulus  intervening  a  monotonous  driving 
session.  Simultaneously  objective  driving- 
parameters  were  derived  from  the  simulator 
(time  to  lane  crossing,  lateral  position  and 
others). 

Results 

The  intervening  stimuli  induced  significant 
group  effects  in  respect  to  EEG  activity  and 
skin  conductance.  There  was  a  decline  of  the 
stimulus  induced  changes  within  several  min¬ 
utes.  Other  physiological  parameters  like 
respiratory  or  cardiac  parameters  were  unaf¬ 
fected.  Data  revealed  a  strong  inter-individual 
variability.  This  applied  also  to  the  perform¬ 
ance  parameters  provided  by  the  simulator. 
Time  to  lane  crossing  and  the  lateral  position 
of  the  car  were  determined  in  different  sce¬ 
narios  and  separated  for  various  types  of 
tracks. 

Conclusion 

The  present  study  revealed  significant 
physiological  group  effects  in  response  to 
intervening  stimuli  during  driving  in  a  real-car 
based  simulator.  EEG  effects,  simulator  data 
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and  first  applications  in  psychiatry  are  dis¬ 
cussed. 

Contact: 

Dr.  Ralph  Mager 
COAT  Basel 
Basel,  4025 
Switzerland 

Phone:  0041613255242 
Fax:  0041613255492 
Email:  ram@coat-basel.com 
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WIRELESS  THERAPY:  MOBILE 
GAMES  AS  A  MEANS  OF  ENGAGING 
ADOLESCENT  CLIENTS 

Mark,  Matthews  BA,  M.  Sc.\  Dr.  Gavin  Do- 
herty\  Dr.  John  Sharry^ 

’Trinity  College  Dublin,  Ireland 
^  Mater  Hospital,  Dublin,  Ireland 

Research  Status:  Currently  in  progress. 

Therapists  have  difficulties  engaging  adoles¬ 
cent  clients  in  therapeutic  activities  between 
sessions.  These  activities  usually  require 
clients  to  prepare  materials  in  advance  of 
sessions  or  chart  their  feelings  between 
them.  Typically  there  is  a  very  low  rate  of 
task  completion  in  these  activities.  This 
might  be  attributed  to  the  use  of  outmoded 
materials  or  methods  which  are  not  concur¬ 
rent  with  adolescent  interests. 

Research  suggests  that  client-centered  ma¬ 
terials  have  a  greater  chance  of  success  in 
engaging  adolescent  clients.  Mobile  gaming 
represents  such  an  approach,  allowing  ac¬ 
cess  to  adolescents'  personal  space,  but 
affording  adolescent  users  persistent  and 
continual  access  to  therapeutic  materials. 
Most  importantly  it  takes  advantage  of  an 
existent  widespread  network  in  use  by  ado¬ 
lescents  and  therefore  requires  no  extra 
hardware  costs.  The  proposed  system  will 
allow  therapists  to  send  personalised  game 
missions  to  clients'  phones  at  specific  agreed 
times  between  sessions.  After  each  level  the 
client  will  receive  a  positive  personal  mes¬ 
sage  from  the  therapist.  The  proposed  game 
will  be  similar  to  the  shoot-em-up  genre  of 
mobile  game.  During  a  session  clients' 


phones  will  synchronise  with  therapists' 
computers  during  sessions  and  allow  thera¬ 
pists  and  clients  to  view  and  discuss  the 
game  information. 

Trials  of  this  system  will  be  run  in  the  Mater 
Hospital  and  several  of  its  associated  clinics 
in  Dublin.  Research  conclusions  will  be 
made  as  a  result  of  pre-  and  post-  question¬ 
naires  for  therapists  and  clients,  detailed 
interviews  with  therapists,  client  attendance 
records,  and  log  data  from  client's  mobile 
games.  A  comparative  study  of  mobile 
games  versus  current  materials  and  ap¬ 
proaches  will  also  be  undertaken. 

It  is  hoped  that  this  research  will  ascertain 
whether  mobile  games  can:  effectively  in¬ 
corporate  therapeutic  elements,  improve  the 
therapeutic  alliance,  help  increase  adoles¬ 
cent  engagement  in  therapeutic  activities 
between  sessions  with  a  therapist  as  well  as 
in  actual  sessions,  increase  the  amount  of 
time  clients  give  between  sessions  to  work¬ 
ing  on  their  issues,  increase  client  task  com¬ 
pletion  rate,  and  reduce  stigma  attached  to 
therapy  through  positive  associations  with 
gaming. 

The  proposed  research  aims  to  evaluate  the 
potential  of  mobile  therapeutic  games  to 
overcome  difficulties  faced  by  therapists  in 
engaging  adolescents  in  therapeutic  activi¬ 
ties  between  therapeutic  sessions.  It  also 
aims  to  discover  whether  this  approach 
might  save  therapist  session  time  by  bring¬ 
ing  therapy  outside  of  sessions  and  into 
adolescents’  everyday  life. 

There  have  been  studies  of  the  potential 
educational  uses  of  mobile  games  to  facili¬ 
tate  cooperative  learning  and  to  examine 
their  potential  as  tools  for  supporting  learn¬ 
ing.  There  has  been  little  research  into  the 
therapeutic  use  of  mobile  technology.  Some 
studies  have  examined  the  use  of  text  mes¬ 
saging  in  client  aftercare,  while  others  have 
tentatively  explored  the  use  of  mobile  de¬ 
vices  to  structure  client  narratives.  To  date, 
there  has  been  a  lack  of  research  exploring 
the  effects  mobile  therapeutic  games  could 
have  on  adolescent  engagement  in  therapy. 
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Background/Problem 

As  many  as  47%  to  58%  of  women  with  can¬ 
cer  report  alopecia  as  the  most  disturbing  an¬ 
ticipated  aspect  of  receiving  chemotherapy.^ 
This  project  develops  and  evaluates  the  effec¬ 
tiveness  of  a  computer  imagining  system  that 
presents  virtually  real  images  of  a  woman 
going  through  the  stages  of  hair  loss/  re¬ 
growth  as  part  of  a  strategy  to  provide  psy¬ 
chosocial  support  to  reduce  distress  associ¬ 
ated  with  chemotherapy-related  alopecia. 

Methods 

Following  Phase  I  development  of  a  prototype 
of  a  computer  imaging  system,  further  devel¬ 
opment  of  a  user-friendly,  touch  screen,  inter¬ 
action  system  that  shows  hair  re-growth  and 
images  sufficient  for  use  to  provide  psychoso¬ 
cial  support  to  women  with  cancer  is  now  un¬ 
der  development.  The  project  has  a  number 
of  tasks  prior  to  the  final  evaluation  of  the  sys¬ 
tem:  Task  1:  Following  Phase  1  focus  groups 
with  a  sample  of  patients  provided  feedback 
on  the  usefulness  of  such  a  system.  Alopecia- 
related  distress  was  also  further  studied.  Task 
2:  A  sample  of  health  care  professionals  were 
interviewed  to  provide  feedback  on  the  accu¬ 
racy  of  the  images  and  usefulness  of  the  sys¬ 
tem.  Task  3:  Complete  and  refine  the  com¬ 
puter  imagining  system.  Task  4:  Conduct  a 
randomized  clinical  trial  comparing  quality  of 
life  between  patients  with  chemotherapy- 
related  alopecia  who  use  the  computer  imag¬ 
ing  system  to  those  who  receive  standard 
care. 

Results 


Tasks  1  and  2  have  been  completed.  The 
system  is  under  Phase  2  development  as 
part  of  Task  3.  Themes  associated  with  che¬ 
motherapy-related  alopecia  include  shock, 
embarrassment,  feeling  unprepared,  and 
anxiety  about  self  and  others  perceptions.  On 
a  scale  of  1  =  not  all  easy  to  10  =  extremely 
easy  to  use,  a  non-clinical  sample  rated  the 
prototype  system  no  lower  than  a  mean  of 
8.66  (SD=1 .28)  on  ease  of  use  in  4  areas. 
Health  care  professionals  in  oncology  rated 
the  realism  of  images  on  a  similar  scale  with 
the  lowest  score  received  a  mean  of  8.94 
(SD=  .93).  Phase  II  is  currently  underway 
with  a  2006  completion  date. 

Conclusion 

Interactive  computer  imaging  systems  using 
virtual  images  have  a  viable  place  in  treat¬ 
ment  for  distress  in  cancer  patients  suffering 
quality  of  life  issues  due  to  alopecia. 
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Since  the  early  1990's,  the  research  on  virtual 
reality  (VR)  applications  for  behavioral  health 
has  progressed  from  exploratory  case  studies 
and  feasibility  studies  to  comparisons  with 
wait-list  and  active  treatment  control 
groups.  This  maturation  of  the  field  has  set 
the  foundation  for  conducting  meta-analytic 
and  review  studies  of  the  research  findings  to 
date.  The  current  investigation  presents  a 
comprehensive  review  of  the  published  VR 
research  in  four  clinical  domains:  (1)  anxiety 
disorders,  (2)  eating  disorders,  (3)  pain  anal¬ 
gesia,  and  (4)  addictions.  In  addition,  a  meta¬ 
analysis  of  andomized  controlled  studies  of 
VR  applications  in  behavioral  health  provides 
a  summary  of  effect  sizes,  making  it  possible 
to  draw  stronger  conclusions  regarding  the 
efficacy  of  VR  interventions  than  significance 
testing  results  alone.  As  such,  this  study 
builds  and  expands  upon  recently  published 
reviews  of  VR  for  anxiety  disorders^'^  and  eat¬ 
ing  disorders."*  The  results  of  this  investigation 
should  be  highly  relevant  and  disseminable 
for  researchers  in  the  area  of  VR  and  the 
more  general  area  of  behavioral  health. 

Method 

The  investigation  consists  of  a  systematic  re¬ 
view  of  the  research  to  February  2005  as  well 
as  a  meta-analysis  of  the  literature.  Utilizing 
Psychinfo  and  Medline,  all  of  the  published 
and  abstracted  studies  in  the  four  clinical  do¬ 
mains  described  earlier  were  identified  and 
reviewed.  Each  study  was  categorized  as  (1) 
a  case  study/uncontrolled  study,  (2)  a  wait¬ 
list/attention  placebo  controlled  study,  or  (3) 
an  active  treatment/standard  care  controlled 
study.  The  meta-analysis  combines  the  effect 
sizes  from  all  controlled  studies  to  produce 
general  indicators  of  the  efficacy  of  VR  inter¬ 
ventions  for  behavioral  health. 

Results 

The  review  component  of  the  investigation 
has  been  completed  of  VR  studies  related  to 
behavioral  health  published  through  Decem¬ 
ber  2004.  Thus  far,  the  data  indicate  that  73 
outcome  studies  have  been  published  across 
the  four  clinical  domains  with  19%  (14)  being 
wait-list  or  attention  placebo  studies,  and  18% 
(13)  active  treatment  or  standard  care  con¬ 
trolled  trials.  A  more  detailed  review  of  each 
clinical  domain  was  done  to  assess  the  scope 
of  the  research  conducted  within  the  specific 
domain.  Preliminary  effect  sizes  computed 


for  select  studies  were  large  in  magnitude, 
and  a  meta-analysis  will  be  performed  with 
the  identified  controlled  VR  studies,  including 
those  published  in  early  2005. 

Conclusions 

The  findings  to  date  suggest  that  VR  for  be¬ 
havioral  health  is  a  maturing  field  that  holds 
considerable  promise  with  continued  re¬ 
search.  Most  of  the  published  studies  are 
either  case  studies  or  uncontrolled  feasibility 
studies,  particularly  for  novel  uses  of  VR  in 
behavioural  health.  Additionally,  some  clini¬ 
cal  domains  have  had  relatively  minimal  at¬ 
tention  (e.g.,  addictions)  as  compared  to  oth¬ 
ers  (e.g.,  anxiety  disorders).  Preliminary  cal¬ 
culations  of  effect  sizes  indicate  high  efficacy 
of  VR,  which  is  not  only  encouraging  for  the 
field,  but  supports  the  need  for  more  con¬ 
trolled  studies  across  clinical  domains.  It  is 
essential  that  there  be  a  better  balance  be¬ 
tween  innovation  and  validation  to  establish 
VR  as  an  effective  intervention  for  behav¬ 
ioural  health. 

Novelty 

To  our  knowledge,  this  is  the  first  statistical 
meta-analysis  of  VR  for  behavioral  health.  In 
addition,  this  review  includes  studies  and 
clinical  domains  that  were  not  included  in 
previous  reviews. 
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Background/Problem 

The  efficacy  of  virtual  reality  (VR)  exposure 
for  the  treatment  of  fear  of  flying  was  con¬ 
firmed  in  more  than  8  controlled  clinical  tri¬ 
als,  Important  goals  of  future  studies  are  the 
evaluation  of  underlying  treatment  processes 
and  the  identification  of  crucial  treatment 
components,  both  in  order  to  optimize  the 
treatment.  The  present  study  was  designed 
to  examine  the  effect  of  motion  simulation  on 
subjective  and  physiological  fear  reactions 
as  part  of  a  VR  exposure  that  also  included 
visual  and  acoustic  stimuli.  Our  hypotheses 
were  that  motion  simulation  results  in 
stronger  initial  subjective  and  physiological 
fear  reactions,  and  as  a  consequence  is  as¬ 
sociated  with  stronger  habituations  of  fear 
responses  within  VR  flights  and  between  VR 
flights.  Furthermore,  we  assumed  that  mo¬ 
tion  simulation  enhances  treatment  efficacy 
as  measured  in  the  fear  of  flying  scale  (FFS). 

Methods  and  Tools 

Twenty  five  participants  with  flight  phobia 
received  a  virtual  reality  exposure  treatment 
including  written  information  about  fear  of 
flying  and  how  to  cope  with  it  (information 
booklet),  one  hour  of  cognitive  therapy,  and 
VR  exposure  (four  flights).  However,  VR  ex¬ 
posure  included  for  twelve  participants  mo¬ 


tion  simulation  while  thirteen  participants 
received  VR  exposure  without  motion  simu¬ 
lation. 

Virtual  flights  consisted  out  of  different  flying 
phases  (start,  flying,  turbulences,  landing) 
and  were  simulated  using  a  head  mounted 
display  (HMD),  a  head  tracking  device  and 
a  motion  platform  to  simulate  accelerations 
and  provide  proprioceptive  stimuli. 

Results 

Overall,  subjective  fear  ratings  as  well  as 
skin  conductance  responses  confirmed  sub¬ 
stantial  fear  of  both  groups  during  VR  expo¬ 
sure.  However,  these  responses  were  sub¬ 
stantially  stronger  and  habituated  slower  in 
the  VR  motion  group  compared  to  the  VR 
no-motion  group.  Nevertheless  and  in  con¬ 
trast  to  network  theories  -  which  suggest 
that  stronger  activation  of  fear  networks 
should  result  in  an  enhanced  treatment  out¬ 
come  -  we  found  no  differences  between 
groups  in  treatment  outcome.  There  was  no 
trend  of  a  superior  treatment  outcome  for 
the  VR-motion  compared  to  the  VR-no- 
motion  group. 

Conclusions  and  Novelty 

The  present  study  helps  to  better  under¬ 
stand  VR  exposure  treatment  and  gives 
hints  for  future  research  to  evaluate  the 
treatment  process.  Based  on  our  results  it 
may  be  speculated  that  treatment  outcome 
is  more  related  to  the  amount  of  habituation 
during  exposure  than  to  the  strength  of  the 
initial  fear  response. 
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Background 

The  potential  for  using  Virtual  Environments 
(VEs)  with  children  with  autistic  spectrum  dis¬ 
orders  (ASDs)  has  been  recognised  by  some 
authors.  The  central  premise  is  that  interac¬ 
tions  in  VEs  may  be  less  threatening  for  chil¬ 
dren  with  ASDs  compared  to  the  Veal  world’ 
because  many  of  the  confusing  inputs  of  real 
world  interaction  can  be  directly  controlled 
and/or  manipulated.  The  present  research 
investigated  whether  single-user  VEs  could 
support  the  learning  of  social  skills  by  adoles¬ 
cents  with  ASDs. 

Method 

A  series  of  quasi-experimental  studies  was 
carried  out  to  investigate  three  main  research 
questions;  would  adolescents  with  ASDs  be 
able  to  use  the  VEs  appropriately,  understand 
VEs  as  representational  devices,  and/or  learn 
new  information  from  VEs  about  social  skills? 

Single-user  VEs  were  developed  to  show  dif¬ 
ferent  social  contexts,  such  as  a  bus  and  a 
cafe,  and  participants  completed  a  number  of 
tasks  such  as  finding  a  place  to  sit,  navigating 
to  the  counter  to  buy  food/drink  and  queuing. 
Non-autistic  comparison  groups  were  in¬ 
cluded  where  possible  and  participants  were 
also  interviewed  about  their  experiences  and 
understanding.  All  sessions  were  videotaped 
for  later  analysis,  such  as  time  taken  to  com¬ 
plete  the  tasks,  navigational  paths,  and  verbal 
responses.  Independent  measures  of  verbal 
(VIQ)  and  non-verbal  (PIQ)  understanding, 
and  Executive  ability  were  also  taken. 

Results 

In  Study  1,  participants  with  ASDs  were  not 
significantly  different  from  a  PIQ-match  group 
in  time  taken  to  complete  VE-based  tasks, 
and  the  majority  understood  the  VE  as  having 
basic  representational  qualities.  Study  2 
showed  that  a  majority  of  participants  with 
ASDs  behaved  in  a  similar  way  to  non-autistic 
comparison  groups  by  treating  the  VE  like  a 
game  in  most  situations.  A  third  of  the  ASD 
group  (4  out  of  12)  showed  substantial  'off- 
task'  behaviour,  which  was  linked  to  low  VIQ 
and  weak  executive  abilities.  Study  3  looked 


at  learning  directly  and  found  that  partici¬ 
pants  improved  in  their  awareness  of  social 
conventions,  both  in  terms  of  performance  in 
the  virtual  cafe  and  in  judgments  and  com¬ 
ments  made  in  relation  to  some  video  clips 
(which  served  as  approximations  of  real  life 
judgments).  Study  4  involved  case  studies 
with  two  participants  and  showed  that  both 
appeared  to  learn  the  target  social  conven¬ 
tions  and  commented  on  how  the  VE  could 
be  relevant  to  their  lives. 

Conclusion 

The  overall  findings  suggest  that  some  ado¬ 
lescents  with  ASDs  can  use,  understand  and 
learn  social  skills/conventions  presented  in 
VEs.  The  main  challenge  now  is  to  reflect  on 
these  findings  in  order  to  shape  future  work. 
Specifically,  there  is  a  need  to  consider  the 
role  of  the  facilitator  in  'scaffolding'  partici¬ 
pants'  use  of  the  VEs.  To  what  extent  can 
facilitation  be  separated  from  the  effect  of  the 
VE  itself?  If  the  aim  is  to  provide  educational 
support,  is  a  separation  of  factors  desirable 
anyway? 

Novelty 

This  series  of  studies  represents  a  system¬ 
atic  and  detailed  addition  to  the  scant  exist¬ 
ing  literature  on  the  use  of  VEs  by  children 
with  ASDs.  The  investigation  into  representa¬ 
tional  understanding,  inclusion  of  carefully 
matched  comparison  groups,  and  considera¬ 
tion  of  clearly  specified  social  learning  objec¬ 
tives  are  particularly  noteworthy  aspects  of 
the  research. 
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From  its  inception  in  1994,  the  Virtual  Reality 
Research  Group  at  the  University  of  East  Lon¬ 
don  recognised  the  enormous  potential  of  vir¬ 
tual  reality  (VR)  as  a  highly  flexible,  rigorously 
controllable,  yet  non-invasive  method  of  di¬ 
rectly  assessing  and  manipulating  brain  activ¬ 
ity  in  order  to  reduce  the  impact  of  brain  dam¬ 
age.  Studies  have  been  conducted  to  address 
whether  individuals  who  have  suffered  brain 
damage  can  use  VR,^  whether  what  is 
learned  in  VR  transfers  to  the  real  world,^  and 
the  implications  of  active  interaction  vs.  pas¬ 
sive  manipulation  within  virtual  environments.^ 
Experiments  have  encompassed  individuals 
who  have  suffered  brain  damage  as  a  result 
of  vascular,''  traumatic,^  degenerative,®  and 
developmental  factors.® 

Memory  impairments  are  one  of  the  most  dis¬ 
abling  consequences  of  brain  injury.^  ®  They 
can  be  problematical  to  assess,  as  conven¬ 
tional  "paper  and  pencil"  neuropsychological 
tests  of  memory  and  cognitive  function  do  not 
adequately  cover  important  aspects  of  every¬ 
day  memory,  such  as  prospective  memory 
(remembering  to  perform  actions  in  the  future) 
and  incidental  memory  (the  encoding  and  re¬ 
call  of  memories  without  any  instructions  from 
a  third  party).®  Paper  and  pencil  tests  have 
also  been  heavily  criticized  as  lacking 
"ecological  validity,"^®  Unfortunately,  it  can  be 
very  difficult  to  achieve  ecological  validity 
without  compromising  experimental  control. 

Virtual  environments  provide  an  elegant  solu¬ 
tion  to  the  probiem  of  deveioping  memory 
tests  and  rehabilitation  strategies  that  are 
valid  from  both  empirical  and  ecological 
standpoints.  In  spite  of  this,  it  is  probably  fair 
to  argue  that  research  into  memory  assess¬ 
ment  and  rehabilitation  in  VR  has  been  under¬ 
exploited.  However,  some  progress  has  been 
made;  for  example,  researchers  have  devel¬ 
oped  VEs  to;  assess  incidental  and  spatial 
memory  via  active  interaction  and  passive 
observation^ enhance  the  restoration  of 
memory  function  in  individuals  with  demen- 
tia^^;  facilitate  in  the  reorganisation  of  memory 
skills  to  use  unimpaired  functions  in  order  to 
train  patients  with  amnesia  to  navigate  around 
a  rehabilitation  ward^®'^'';  encourage  the  de¬ 
velopment  of  spatial  memory  in  children 


whose  physical  disabilities  restrict  their  mo- 
bility^®'^®'^  ,  and  motivate  patients  with  trau¬ 
matic  brain  injury  (TBI)  to  participate  in  exer¬ 
cise  in  order  to  stimulate  memory  remedia¬ 
tion.'®  There  has  also  been  some  preliminary 
research  on  prospective  memory  in  patients 
who  have  suffered  a  stroke'®  and  older  indi¬ 
viduals.^®  Finally,  using  VR  together  with 
fMRI  offers  exciting  opportunities  to  address 
the  neurophysiological  basis  of  different 
types  of  memory.^These,  and  other  possibili¬ 
ties,  are  discussed. 

This  paper  evaluates  the  current  state  of  re¬ 
search  on  the  applications  of  VR  to  memory 
assessment  and  rehabilitation,  proposes 
some  directions  for  future  research,  de¬ 
scribes  some  recent  work  on  VR  and  pro¬ 
spective  memory  being  conducted  at  UEL, 
and  suggests  a  strategy  for  turning  our  re¬ 
search  findings  into  real  benefits  for  patients. 
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Research  status:  In  Progress 

Background 

Sleepiness  in  the  work  and  traffic  environment 
can  cause  severe  accidents.  Sleepiness  is 
either  the  consequence  of  sleep  loss  or  sleep 
deprivation  or  of  a  sleep  disorder.  Car  acci¬ 
dents  have  more  fatal  outcomes  at  night  than 
during  the  day.  Drowsy  or  sleepy  drivers  and 
workers  at  monitoring  tasks  do  have  a  slower 
response  to  react  adequately  to  alarms  or 
dangerous  situations.  Therefore  there  is  a 
need  to  monitor  sleepiness  in  such  circum¬ 
stances. 

Methods:  A  European  project  uses  existing 
technologies  and  sensors  to  develop  new  in¬ 
tegrated  systems  in  order  to  detect  a  lower 
vigilance  at  an  early  stage.  Definitions  for  the 
different  levels  of  sleepiness  were  developed 
based  on  monitoring  of  brain  function^  and 
autonomous  nervous  system  functions.^  The 
most  sensitive  parameters  will  be  chosen  in 
order  to  monitor  sleepiness  without  impair¬ 
ment  of  the  driver  or  worker.  An  alarm  or 
other  adequate  feedback  will  be  developed  in 
order  to  create  awareness  to  the  driver  or 
worker.  The  same  sensors  are  also  used  for 
the  medical  applications  in  order  to  diagnose 
sleep  disorders.  The  specific  challenge  is  to 
detect  sleep  disorders  at  an  early  stage  in 
order  to  initiate  prevention  programs. 

Results 

A  set  of  important  signals  for  the  identification 
of  sleepiness  has  been  identified.  The  electro¬ 
encephalogram,  the  electromyogram,  and 
heart  rate  are  sensitive  signals.  Among  all 
sleep  disorders  the  most  prevalent  sleep  dis¬ 
orders  are  addressed.  These  are  insomnia, 
sleep  disordered  breathing,  periodic  limb 
movements  during  sleep,  and  narcolepsy. 
Application  scenarios  for  work  environment  as 
well  as  for  diagnostic  and  treatment  ap¬ 
proaches  were  defined. 

Conclusions 

The  definitions  reached  up  to  now  allow  to 
specify  systems  for  use  in  the  work  and  traffic 
field  as  well  as  in  the  medical  environment. 


The  new  developed  sensors  and  analysis 
algorithms  are  then  applied  in  sleep  laborato¬ 
ries  and  intensive  care  medicine  with  moni¬ 
toring  demands.  Wireless  data  transmission 
based  on  body  area  network  technologies  is 
of  major  importance  for  these  applications. 

Novelty 

Existing  applications  in  sleep  diagnosis  are 
based  on  wired  technology  and  are  restricted 
to  hospital  based  sleep  laboratories.  The  new 
systems  will  allow  a  medical  diagnosis  in  out¬ 
patient  settings  with  the  same  signal  quality 
as  reached  in  the  hospital  until  now.  The  sys¬ 
tematic  approach  to  signals  and  parameters 
will  enable  the  development  of  improved  al¬ 
gorithms  for  diagnosis.  Sensors  technology 
developed  for  automotive  purposes  can  be 
reused  in  medical  diagnostic  environment. 
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The  market  of  mobile  phones  is  increasing 
quickly  and  now  account  for  shipments  of 
over  400  million  units  per  year.  The  mobiles  of 
new  generation  bring  much  functionality,  as 
PDA,  cameras,  gaming  devices,  or  media 
players.  These  innovations  will  grow  in  the 
next  years,  so  a  new  generation  of  hardware- 
accelerated  mobile  devices  will  soon  be 
joined  by  a  suite  of  emerging  3D  software 
standards  that  gives  developers  the  ability  to 
create  interactive  content  and  applications 
that  haven’t  been  possible  before. 

In  this  context,  mobile  narratives  are  emerg¬ 
ing  as  possible  applications.  Mobile  narratives 
are  interactive  multimedia  experiences  imple¬ 
mented  on  mobile  devices,  in  which  the  narra¬ 
tive  component  is  a  critical  aspect  to  create  a 
feeling  of  presence  and  engagement. 


Schedulef  and  the  STAI  {State  Trait  Anxiety 
Inventory).^ 

The  final  results  will  be  presented  at  the  Cy¬ 
bertherapy  2005  conference. 
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The  mobile  narratives,  through  the  link  be¬ 
tween  the  feeling  of  presence  and  the  emo¬ 
tional  state,  may  be  used  to  improve  the 
mood  state  in  the  users.  In  particular,  mobile 
narratives  may  be  used  for  both  modifying  the 
user’s  mood  and  helping  him  to  relax,  en¬ 
hancing  his  well-being. 

Relaxing  is  a  difficult  activity,  especially  in 
everyday  life.  Many  of  us  are  usually  involved 
in  stressful  situations  far  away  from  the  con¬ 
text  required  by  the  classical  relaxation  tech¬ 
niques.  In  this  study  -  supported  by  the  Italian 
“NeuroTiv”  MIUR  FIRB  research  project^  we 
developed  a  mobile  narrative  for  UMTS 
phones  to  be  used  by  commuters  to  improve 
their  well-being.  The  narrative,  based  on  a  trip 
in  a  desert  tropical  island,  relaxes  the  user 
through  different  audio-visual  experiences. 
They  include  the  exploration  of  the  island  and 
different  relaxation  techniques  based  on  the 
Progressive  Relaxation  protocol  developed  by 
Jacobson.^ 

The  experimental  sample  is  composed  by  90 
commuting  students,  aged  between  20  an  24, 
randomly  divided  in  3  different  conditions:  the 
experimental  group,  which  experienced  the 
mobile  narrative  during  their  daily  train  trip;  a 
first  control  group,  which  experienced  a  relax¬ 
ing  new  age  DVD  during  their  train  trip;  and  a 
second  control  group,  which  experienced  the 
train  trip  only. 
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Research  status:  In  progress 

Sexual  preferences  are  usually  assessed 
either  by  resorting  on  sexual  responses  re¬ 
corded  with  a  penile  plethysmograph  or  by 
using  visual  reaction  time  from  stimuli  with 


Each  subject  was  submitted  before  and  after 
the  experience  to  the  following  questionnaires 
the  PANAS  {Positive  and  Negative  Affects 
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sexual  content.  However,  numerous  short¬ 
comings  come  with  penile  plethysmography 
and  visual  reaction  time,  and  notably  volun¬ 
tary  erectile  control  and  not  paying  attention 
to  stimuli,  which  are  important  threats  to  the 
internal  validity  of  these  assessment  proce¬ 
dures.  In  order  to  get  round  these  limitations, 
we  developed  a  method  that  controls  gazing 
activity  relative  to  sexual  avatars.  A  study  is 
conducted  using  our  method  that  is  aiming  at 
assessing  sexual  preferences  with  sex  offend¬ 
ers  and  control  subjects  interacting  with  sex¬ 
ual  avatars. 

Background 

Sexual  preferences  are  usually  assessed 
either  by  resorting  on  sexual  responses  re¬ 
corded  with  a  penile  plethysmograph  or  by 
using  visual  reaction  time  from  stimuli  with 
sexual  content.  However,  numerous  short¬ 
comings  come  with  penile  plethysmography 
and  visual  reaction  time,  and  notably  volun¬ 
tary  erectile  control  and  not  paying  attention 
to  stimuli,  which  are  important  threats  to  the 
internal  validity  of  these  assessment  proce¬ 
dures.  In  order  to  get  round  these  limitations, 
we  developed  a  method  that  controls  gazing 
activity  relative  to  sexual  avatars.  A  study  is 
conducted  using  our  method  that  is  aiming  at 
assessing  sexual  preferences  with  sex  offend¬ 
ers  and  control  subjects  interacting  with  sex¬ 
ual  avatars. 

Methods 

Our  method  relies  upon  a  technological  setting 
including  what  is  usually  necessary  to  present 
virtual  environments  in  immersion  plus  equip¬ 
ments  dedicated  to  eye-movements  tracking 
from  within  a  head  mounted  display  (HMD).  A 
special  mounting  built  from  a  monocular  infra¬ 
red  eye-tracking  system  combined  within  a 
binocular  HMD  is  used  to  track  eye- 
movements.  Head-movements  are  recorded 
from  a  magnetic  tracking  system  rendering  the 
6  degrees-of-freedom  (DOF)  of  translation  and 
rotation.  Our  method  performs  gaze  analysis 
by  the  way  of  virtual  measurement  points 
(WiP)  placed  on  virtual  objects  for  the  analysis 
of  eye-movements  in  relation  to  specific  fea¬ 
tures  of  these  objects.  Gaze  radial  angular  de¬ 
viation  (GRAD)  from  the  VMP  is  given  by  the 
combinations  of  the  6  DOF  developed  by 
head-movements  and  the  x  and  y  coordinates 
rendered  by  the  eye-tracking  system.  The  VMP 
is  locked  to  virtual  objects  and  moves  jointly 
with  them.  While  the  variations  in  the  6  DOF 


developed  by  head-movements  define 
changes  in  the  global  scene  presented  in  the 
HMD,  the  2  DOF  given  by  the  eye-tracking 
device  allow  the  computation  of  the  exact  po¬ 
sition  of  the  line  of  sight  relative  to  the  VMP. 
As  other  physiologic  signals,  we  also  measure 
the  subject’s  distance  from  the  VPR,  the  pupil 
size  diameter,  the  blinking  response  and  the 
erectile  response  recorded  with  a  penile 
plethysmograph.  The  3D  stimuli  that  we  used 
are  animated  naked  human  models. 

Results 

Preliminary  results  from  sex  offenders  and 
control  subjects  will  be  presented.  These 
results  will  consist  in  analyses  performed  on 
time  series  coming  from  oculomotor  and 
other  physiologic  responses  recorded  in  im¬ 
mersion. 

Novelty 

The  novelty  of  this  method  for  assessing  sex¬ 
ual  preferences  was  acknowledged  at  the 
last  Conference  of  the  Association  for  the 
Treatment  of  Sexual  Abusers  held  October 
2004  in  Albuquerque  (NM).  These  results 
inscribe  themselves  in  the  extension  of  our 
past  studies  whose  goal  was  mainly  to  make 
sense  of  the  perceptual  and  motor  dimen¬ 
sions  of  virtual  immersion.^'®  They  also  pave 
the  way  to  interactive  behavior  and  percep¬ 
tion  modification  therapies  based  on  eye¬ 
tracking  recorded  in  VR.®  ’^ 
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VIRTUAL  REALITY  IN  EATING  DISOR¬ 
DERS  AND  OBESITY:  STATE  OF  THE 
ART  AND  FUTURE  DIRECTIONS 

Prof.  Giuseppe  Riva,  Ph.D. 

Applied  Technology  for  Neuro-Psychology  Lab 
Istituto  Auxologico  Italiano 

The  first  study  describing  the  possible  use  of 
virtual  reality  in  the  treatment  of  eating  disor¬ 
ders  and  obesity  was  published  in  1995. 
Since  then,  different  researchers  have  been 
studying  the  possible  advantages  of  virtual 
reality  as  experiential  technique  able  to  mod¬ 
ify  the  body  image,  self-efficacy  and  emo¬ 
tional  control  of  the  patient. 


This  presentation  will  review  the  state  of  the 
art  in  this  research  field.  In  particular,  it  will 
discuss  the  result  coming  from  different  con¬ 
trolled  trials  comparing  this  approach  with 
competing  techniques. 

Further,  the  presentation  will  address  the 
future  directions  of  this  method  focusing  on 
the  possible  opportunities  coming  from  Inter¬ 
net  and  mobile  devices. 


Presenter:  Albert  “Skip”  Rizzo  PhD 

GAMES  FOR  HEALTH:  GAME-BASED 
VIRTUAL  REALITY  APPLICATIONS 
FOR  MENTAL  DISORDERS  &  REHA¬ 
BILITATION 

Albert  "Skip"  Rizzo',  Jarrell  Pair,  Galen  Buck- 
waiter,  Jeff  Gold,  Carolee  Winstein,  Margaret 
McLaughlin,  Ken  Graap,  Brenda  Wiederhold 

’  University  of  Southern  California  Institute  for 
Creative  T  echnologies 

The  rapidly  growing  popularity  of  digital  gam¬ 
ing  has  driven  advances  in  game-based 
technology  that  has  now  added  new  momen¬ 
tum  to  the  vision  of  creating  immersive  and 
interactive  experiential  opportunities  for  a 
wide  range  of  human  needs  beyond  just  the 
entertainment  value  that  come  from  playing 
games.  In  addition  to  the  readily  intuitive  ap¬ 
plication  of  game-based  earning  for  educa¬ 
tion  and  training  purposes  (Prensky,  2001), 
the  mental  health  and  rehabilitation  fields 
stand  to  substantially  benefit  from  advances 
in  this  area.  The  integration  of  game  technol¬ 
ogy  and  experiences  with  VR-based  ap¬ 
proaches  for  clinical  assessment,  treatment, 
and  rehabilitation  offers  powerful  options  that 
could  revolutionize  standard  practices  in 
these  fields.  Thus  far,  the  integration  of  gam¬ 
ing  features  into  a  VE  has  been  reported  to 
enhance  motivation  in  adult  clients  undergo¬ 
ing  physical  and  occupational  therapy  follow¬ 
ing  a  stroke  (Jack  et  al,  2001 ;  Kizony,  Katz  & 
Weiss,  2003)  and  to  reduce  acute  pain  dur¬ 
ing  medical  procedures  (Hoffman  et  al., 

2000).  As  well,  Strickland  (2001)  reports  that 
children  with  autism  were  observed  to  be¬ 
come  very  engaged  in  the  VR  safety  training 
applications  that  she  has  developed  which 
incorporate  game  features.  Further  anecdotal 
observations  suggest  that  children  diagnosed 
with  Attention  Deficit  Hyperactivity  Disorder 
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often  have  a  fascination  for  the  type  of  stimu¬ 
lus  environments  that  are  presented  in 
computer/video  games  (Greenhill,  1998).  Par¬ 
ents  are  often  puzzled  when  they  observe 
their  children  focusing  on  computer  games 
intently,  while  teacher  reports  indicate  inatten¬ 
tion  in  the  classroom.  Additionally,  in  the  pre¬ 
senter's  previous  clinical  experience,  it  was 
anecdotally  observed  that  certain  young  adult 
traumatic  brain  injury  clients,  who  had  diffi¬ 
culty  maintaining  concentration  on  traditional 
cognitive  rehabilitation  tasks,  would  easily 
spend  hours  at  a  time  playing  the  computer 
game  "SimCity.”  These  observations  suggest 
that  designers  of  clinical  assessment  and 
treatment  tasks  could  benefit  from  examining 
the  formulas  that  commercial  game  develop¬ 
ers  use  in  the  creation  of  interactive  digital 
games.  These  formulas  govern  the  flow  and 
variation  in  stimulus  pacing  that  provide  link¬ 
age  to  a  progressive  reward  and  goal  struc¬ 
ture.  When  delivered  within  a  highly  interac¬ 
tive  graphics-rich  environment,  users  are 
commonly  observed  to  become  extremely 
engaged  in  this  sort  of  game  play.  This  pres¬ 
entation  will  describe  a  variety  of  ongoing  pro¬ 
jects  at  use  that  integrate  game  technology 
with  VR-based  applications  for  the  assess¬ 
ment,  treatment  and  rehabiiitation  of  cogni¬ 
tive,  psychological,  emotional,  and  physical 
disorders.  These  projects  have  focused  on 
the  development  of  applications  for: 

1.  Attention  process  assessment  in  children 
with  ADHD 

2.  Pain  distraction  in  children  undergoing 
painful  medical  procedures  and  chemother¬ 
apy 

3.  Exposure  therapy  in  returning  Iraq  War  vet¬ 
erans  with  PTSD 

4.  Motor  rehabilitation  for  persons  with  central 
nervous  system  dysfunction  (i.e.  stroke,  brain 
injury) 

These  applications  have  leveraged  the  assets 
that  have  come  from  advances  in  both  game 
engine  technology  and  from  the  unique  ex¬ 
perience  that  is  fostered  from  the  psychologi¬ 
cally  involving  act  of  game  play.  Such  USC 
initiatives  may  provide  hope  that  innovative 
methods  can  be  created  to  advance  these 
areas  that  have  long  been  mired  in  the  meth¬ 
ods  of  the  past.  My  presentation  will  begin 
with  an  introduction  on  how  and  why  game 
assets  can  be  used  to  create  applications  in 
mental  health  and  rehabilitation  and  then  will 
review  the  data/status  of  our  various  projects. 
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FROM  TRAINING  TO  TOY  TO  TREAT¬ 
MENT:  PRELIMINARY  RESULTS 
FROM  A  VR  THERAPY  APPLICATION 
FOR  IRAQ  WAR  MILITARY  PERSON¬ 
NEL  WITH  COMBAT-RELATED  PTSD 

Albert  "Skip"  Rizzo\  Jarrell  Pair,  Peter  J. 
McNerney,  Ernie  Eastiund,  Brian  Manson, 

Jon  Gratch,  Randy  Hill,  Michael  Roy*,  Bill 
Swartout,  Ken  Graap  &  Brenda  Wiederhold 
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Creative  Technologies 

In  1997,  researchers  at  Georgia  Tech  re¬ 
leased  the  first  version  of  the  Virtual  Vietnam 
VR  scenario  for  use  as  a  graduated  exposure 
therapy  treatment  for  PTSD  in  Vietnam  veter¬ 
ans.  This  occurred  over  20  years  following 
the  end  of  the  Vietnam  War.  During  that  in¬ 
terval,  in  spite  of  valiant  efforts  to  develop 
and  apply  traditional 

psychotherapeutic  approaches  to  PTSD,  the 
progression  of  the  disorder  in  some  veterans 
severely  impaired  their  functional  abilities 
and  quality  of  life,  as  well  as  that  of  their  fam¬ 
ily  members  and  friends.  Prior  to  the  avail¬ 
ability  of  VR  therapy  applications,  the  existing 
standard  of  care  for  PTSD  was  imaginal  ex¬ 
posure  therapy.  Such  treatment  typically  in¬ 
volves  the  graded  and  repeated  imaginal 
reliving  of  the  traumatic  event  within  the 
therapeutic  setting.  This  approach  is  believed 
to  provide  a  low-threat  context  where  the 
patient  can  begin  to  therapeutically  process 
the  emotions  that  are  relevant  to  the  trau¬ 
matic  event  as  well  as  de-condition  the  learn¬ 
ing  cycle  of  the  disorder  via  a  habituation/ 
extinction  process.  While  the  efficacy  of 
imaginal  exposure  has  been  established  in 
multiple  studies  with  diverse  trauma  popula¬ 
tions  (Rothbaum  &  Schwartz,  2002),  many 
patients  are  unwilling  or  unable  to  effectively 
visualize  the  traumatic  event.  With  this  his¬ 
tory  in  mind,  the  USC  Institute  for  Creative 
Technologies  (ICT)  has  initiated  a  project 
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that  is  creating  an  immersive  virtual  environ¬ 
ment  system  for  the  treatment  of  Iraq  War 
military  personnel  diagnosed  with  combat- 
related  PTSD.  The  treatment  environment  is 
based  on  a  creative  approach  to  recycling 
virtual  assets  that  were  initially  built  for  a  com¬ 
bat  tactical  simulation  scenario  entitled  Full 
Spectrum  Command,  and  later  licensed  to 
create  the  commercially  available  X-Box 
game,  Full  Spectrum  VVarrior. 

Thus  far  we  have  created  a  prototype  virtual 
environment  designed  to  resemble  a  Middle 
Eastern  city.  We  intend  to  use  this  VE  for  ini¬ 
tial  clinical  and  patient  user  testing  to  gather 
feedback  to  refine  the  city  scenario  and  for 
the  future  expansion  of  the  system  to  include 
other  relevant  scenarios  (e.g.  outlying  village 
and  desert  scenes,  etc.)  and  user  perspec¬ 
tives.  We  have  also  created  an  initial  version 
of  a  "wizard  of  oz"  type  clinical  interface.  This 
interface  is  a  key  element  in  that  it  will  provide 
a  clinician  with  the  capacity  to  customize  the 
therapy  experience  to  the  individual  needs  of 
the  patient  by  placing  them  in  VE  locations 
that  resemble  the  setting  in  which  the  trau¬ 
matic  events  initially  occurred.  The  interface 
will  also  allow  the  therapist  to  gradually  intro¬ 
duce  and  control  "trigger"  stimuli  in  the  envi¬ 
ronment  in  real  time  as  is  required  to  foster 
the  anxiety  modulation  needed  for  therapeutic 
habituation.  This  presentation  will  discuss  the 
vision,  rationale  and  user-centered  develop¬ 
ment  status  of  the  Full  Spectrum  PTSD  treat¬ 
ment  system  that  is  currentiy  in  progress  at 
the  ICT.  As  well,  initial  results  will  be  pre¬ 
sented  from  ongoing  user-centered  design 
trials  that  are  part  of  the  development  of  this 
immersive  VR  application  for  this  highly  rele¬ 
vant  mental  healthcare  problem. 
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COMPARATIVE  STUDY  OF  IMMER¬ 
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Background 

Virtual  reality  (VR)  can  become  a  useful  tool 
to  overcome  obesity,  especially  for  young¬ 
sters,  considering  its  similarity  to  video 
games.  This  comparative  study  aim  to  as¬ 
sess  what  type  of  immersive  display  is  pre¬ 
ferred  by  youngsters  for  using  a  virtual  bicy¬ 
cle.  This  is  the  first  step  in  the  development 
of  a  program  to  enhance  motivation  in  obese 
youngsters  to  exercise. 

Method/Tools 

Twenty-one  healthy  participants  aged  10-18 
years  old  (mean  age  =  12.5,  52.4%  male) 
were  immersed  three  times  (three  minutes 
each)  in  a  virtual  environment  using  a  differ¬ 
ent  type  of  technology  for  each  immersion 
(HMD,  projector,  television).  The  order  of 
immersive  technology  was  randomly  as¬ 
signed.  The  hardware  used  was  a  10-speed 
bicycle  connected  to  an  1-Magic®  VR  trainer 
steering  with  an  integrated  tracker  connected 
to  a  Pentium  III®  866  Mhz  PC.  The  HMDs 
used  were  an  l-Visor®  for  half  of  the  partici¬ 
pants  and  an  l-Glass®.  An  Eiki®  LCD  multi- 
media  projector  and  a  Sony®  27-inch  televi¬ 
sion  were  used.  Pre-post  measures  of  Im¬ 
mersive  Tendencies  and  the  Simulator  Sick¬ 
ness  were  completed.  Subjective  measures 
were  administered  after  each  immersion  for 
presence,  cybersickness,  and  preferences. 

Results 

Participant  significantly  preferred  the  HMD 
over  the  projector  and  the  television.  They 
felt  more  "present"  using  the  HMD  but  also 
more  cybersickness.  No  significant  difference 
was  found  in  the  preference  for  the  television 
vs.  projector.  Immersive  Tendencies  corre¬ 
lates  significantly  with  presence  when  HMD 
was  used. 

Conclusion 

Results  indicate  that  levels  presence,  prefer¬ 
ence,  and  cybersickness  are  higher  when 
using  a  HMD.  Considering  the  purpose  of 
this  study,  cybersickness  has  to  be  acknowl¬ 
edge  since  obese  youngsters  will  use  the 
virtual  bicycle  for  a  longer  periods.  The  pro¬ 
jector  and  television  still  had  a  high  level  of 
interest  for  participants  and  low  cybersick- 
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ness.  LCD  projectors  are  the  best  technology 
to  use  as  an  immersive  display  for  virtual  ex¬ 
ercises. 
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DEVELOPMENT  OF  A  BRAIN- 
COMPUTER  INTERFACE  (BCI)  BASED 
ON  VIRTUAL  REALITY  TO  IMPROVE 
TRAINING  TECHNIQUES 

Ricardo  Ron-Angevin,  PhD  candidate, 

Antonio  Diaz-Estrella,  Arcadio  Reyes- 
Lecuona 

Departamento  de  Tecnologia  Electrbnica, 
Universidad  de  Malaga 

Research  Status:  In  progress. 

Background/Problem 

A  brain-computer  interface  (BCI)  is  based  on 
the  analysis  of  the  electroencephalographic 
signals  (EEG),  recorded  during  certain  mental 
activities,  to  control  an  external  device.  One  of 
its  main  uses  could  be  in  the  field  of  medicine 
and  especially  in  rehabilitation.  It  could  help  to 
establish  a  communication  and  control  chan¬ 
nel  for  people  with  serious  motor  function 
problems  but  without  brain  function  disorder. 

Performance  of  BCI  will  depend,  especially, 
on  the  ability  of  subjects  to  control  their  EEG 
patterns  being  necessary  to  provide  suitable 
training.  The  vast  majority  of  research  in  the 
BCI  field  puts  emphasis  on  the  importance  of 
developing  training  methods  based  on  bio¬ 
feedback  techniques  that  would  improve  hu¬ 
man  performance. 

Training  in  BCI  consists  on  carrying  out  some 
trials,  repeated  times,  being  very  important  to 
provide  some  type  of  feedback  allowing  sub¬ 
jects  to  see  their  progress  [1].  Training  proto¬ 
cols  must  not  be  chosen  at  random.  On  the 
contrary,  in  many  cases,  they  must  adapt  to 


the  subject  to  be  effective.  To  design  a  suit¬ 
able  training  protocol,  it  would  be  interesting 
to  study  possible  effects  that  diferent  type  of 
feedback  can  have  on  subjects.  Feedback 
may  help  subjects  to  improve  their  EEG  con¬ 
trol,  but  sometimes  its  effects  can  be  frustrat¬ 
ing.  Besides, 

conventional  systems  of  feedback,  such  as 
cursor  control  or  horizontal  bar  extension  can 
prove  somewhat  boring,  leading  to  a  lack  of 
motivation. 

Method/Tools 

To  avoid  this  problem,  feedback  must  be  at¬ 
tractive,  motivating  subjects  to  control  their 
EEG  signals.  For  this,  a  good  option  is  the 
use  of  techniques  based  on  virtual  reality  [2], 
combining  3D  display,  sound  and  isolation. 
Using  these  techniques,  a  more  natural  inter¬ 
action  can  be  achieved,  isolating  subjects 
from  distraction,  and  providing  a  more  im¬ 
mersed  and  motivating  environment.  Based 
on  virtual  reality  techniques,  a  new  type  of 
BCI  system  to  enable  to  establish  the  appro¬ 
priate  feedback  allowing  subjects  to  get  bet¬ 
ter  control  of  EEG  signals,  is  developed. 

Results 

This  BCI  allows  to  set  parameters  related  to 
training  paradigm  timing  (-  duration,  repeti¬ 
tiveness  and  pause  between  trials,  -  duration 
of  the  feedback  period,  -  duration  of  the  ses¬ 
sion),  the  type  of  feedback  (continuous  or 
discrete  feedback)  and  the  presentation  of 
feedback  selecting  different  scenes.  Due  to 
trial  repetitiveness,  interface  were  designed 
to  be  as  appealing  and  motivating  as  possi¬ 
ble.  It  was  decided  to  submit  subjects  to  a 
more  familiar  environment,  such  as  control¬ 
ling  a  car  to  avoid  crashing  it  into  a  wall,  to 
avoid  logs  in  the  road,  or  to  reach  a  ramp 
that  would  make  the  car  jump.  Preliminary 
results  show  how  this  new  system  can  be 
useful  to  study  and  to  improve  training  proto¬ 
col  in  BCI. 

Conclusions 

The  new  BCI  system  is  very  promising,  pro¬ 
viding  an  attractive  training  to  the  subject  and 
allowing  to  adapt  training  protocols  to  be  ef¬ 
fective. 

Novelty/Discussion 
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One  of  the  differences  of  this  new  system, 
compared  to  conventional  feedbacks,  such  as 
cursor  control  or  horizontal  bar  extension,  is 
the  use  of  virtual  reality  to  provide  feedback 
with  more  immersive  and  motivating  effects. 
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Continuing  medical  education  has  historically 
been  provided  largely  through  didactic  class¬ 
room-style  teaching.  However,  educators 
have  identified  this  as  one  of  the  least  effec¬ 
tive  methods  of  adult  education.  Adult  learn¬ 
ers  are  far  more  likely  to  retain  new  informa¬ 
tion  that  is  required  through  experiential  or 
self-directed  learning.  In  addition,  many 
young  adults  today  havecome  of  age  with  sig¬ 
nificant  experience  with  computer-based  or 
“video”  games.  Comfort  with  new  technolo¬ 
gies  renders  new  physicians  and  physicians- 
in-training  ripe  for  medical  education  that  en¬ 
gages  them  through  such  technologies.  Simi¬ 


larly,  it  provides  an  opportunity  to  enhance 
treatment  of  their  patients  through  such  mo¬ 
dalities. 

We  report  our  experience  with  the  use  of 
standardized  patients  (SPs)  in  educating 
physicians  regarding  the  diagnosis  and  treat¬ 
ment  of  biological  and  chemical  warfare 
agent  exposure.  We  trained  professional 
actors  to  serve  as  SPs  representing  expo¬ 
sure  to  the  biological  agents  anthrax,  small¬ 
pox,  botulinum  toxin,  and  staphylococcus 
enterotoxin  B  (SEB).  We  conducted  work¬ 
shops  at  two  national  medical  meetings,  with 
workshop  participants  rotating  in  small 
groups  through  a  series  of  teaching  stations 
where  they  were  able  to  perform  a  medical 
history  and  physical  examination,  order  tests, 
make  a  diagnosis,  and  establish  a  manage¬ 
ment  plan.  Workshop  staff  physicians  then 
conducted  discussions  and  emphasized  key 
teaching  points.  In  addition,  we  trained  SPs 
to  simulate  a  mass  casualty  (MASCAL)  inci¬ 
dent  involving  a  terrorist  spraying  a  liquid  on 
a  city  bus,  once  using  sarin,  and  in  another 
case  using  hydrochloric  acid.  All  workshop 
participants  worked  together  to  treat  MAS¬ 
CAL  victims,  followed  by  debriefing  and  dis¬ 
cussion. 

We  also  describe  a  current  project  to  develop 
computer-based  interactive  modules  of  simu¬ 
lated  patients  exposed  to  smallpox,  anthrax, 
and  other  biological  and  chemical  agents. 

The  nature  of  the  program  allows  the  user  to 
go  through  the  simulation  repeatedly,  with  a 
different  presentation  each  time,  based  partly 
on  the  questions  they  pose  and  the  way  they 
ask  them,  as  well  as  by  chance.  The  pro¬ 
gram  provides  feedback  on  each  question 
and  on  the  overall  conduct  of  the  interview. 

The  advantages  and  disadvantages  of  each 
of  these  methods  will  be  discussed. 

Contact: 

Michael  J.  Roy,  MD,  MPH 
Professor  of  Medicine 
Uniformed  Services  University 
Bethesda,  MD 
USA 
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TRAINING  BLIND  CHILDREN  TO  DE¬ 
VELOP  MATHEMATICS  SKILLS 
THROUGH  AUDIO 


290 


CyberTherapy  2005  Abstracts 


Jaime  Sanchez,  Ph  D. 

University  of  Chile 

It  is  well  known  that  mathematics  allow  us  to 
abstract  and  understand  our  surrounding 
world.  Thus  an  early  training  to  people  in 
mathematics  can  help  to  develop  better  learn¬ 
ing  skills.  People  with  visual  disabilities  can 
not  learn  mathematics  easily.  Mathematics  is 
developed,  represented,  and  communicated 
basically  through  visual  means  such  as  in  al¬ 
gebraic  expressions,  graphs,  and  geometric 
figures.  A  few  studies  have  approached  this 
issue  by  studying  new  interfaces  for  mathe¬ 
matics  learning.^ 

This  research  analyses  how  audio-based 
technology  can  be  used  as  a  powerful  tool  to 
train  blind  children  to  learn  mathematics  con¬ 
cepts.  We  introduce  and  analyze  three  audio- 
based  software  products  to  develop  mathe¬ 
matics  skills. 

AudioMath  is  an  audio-based  interactive  ap¬ 
plication  that  mimics  the  social  board  game 
Memory.®'®  The  child  has  to  open  pairs  of  to¬ 
kens  in  a  board  with  several  levels  of  difficulty 
to  find  the  corresponding  pair  of  tokens  in  ac¬ 
cordance  with  the  mathematical  content  such 
as  correspondence  and  equivalency  relation¬ 
ships,  memory  development,  and  differentiat¬ 
ing  tempo-spatial  notions. 

Theo  &  Seth  is  an  audio-based  interactive 
virtual  environment  to  enhance  learning  of 
number  and  basic  operations  with  different 
levels  of  complexity.  The  metaphor  used  re¬ 
sembles  a  grange  with  two  major  virtual  envi¬ 
ronments;  the  kitchen  and  the  henhouse.  The 
kitchen  covers  cardinality  including  the  num¬ 
ber,  the  position  in  the  numerical  straight  line, 
information  about  the  antecessor  and  succes¬ 
sor;  and  the  introduction  of  ordinal  numbers. 
The  henhouse  is  a  virtual  space  where  learn¬ 
ers  learn  how  to  do  sum  and  rest. 

AudioGeometry  is  based  on  Van  Hiele's 
model  of  geometric  thinking  and  serve  as  a 
guide  for  instruction  and  evaluation  of  geo¬ 
metric  skills.  The  software  is  implemented 
with  a  useful  scheme  to  organize  content 
(plane  figures,  area,  and  perimeter).  Audio- 
Geometry  consists  of  identifying,  recognizing, 
and  naming  polygons  and  related  elements  as 
well  as  measuring  geometric  figures  and  cal¬ 
culating  areas  and  perimeters  of  plane  geo¬ 


metric  figures. 

We  have  tested  these  audio-based  products 
with  twenty  blind  learners  from  7  to  15  years 
old  in  a  school  setting  by  using  tests  for 
mathematics  knowledge  and  thinking,  and 
audio  memory.  We  implemented  a  case 
study  by  doing  cognitive  introspection  during 
and  after  each  task.  Children  were  exposed 
to  software  with  formal  activities  and  solved 
related  cognitive  tasks  during  six  months.  We 
have  preliminary  evidenced  that  the  use  of 
audio-based  software  with  related  cognitive 
tasks  can  help  to  develop  mathematics  think¬ 
ing  and  knowledge.  Blind  children  have 
shown  positive  gains  in  learning  addition, 
subtraction,  multiplication,  and  division  after 
interaction  with  audio-based  virtual  environ¬ 
ments.  They  have  improved  the  development 
and  use  of  mathematics  skills. 

A  usability  evaluation  was  also  implemented 
during  and  after  the  implementation.  Children 
enriched  the  software  after  evaluating  differ¬ 
ent  modules  during  implementation  and  help¬ 
ing  us  to  map  their  mental  models. 

Finally,  this  study  adds  preliminary  knowl¬ 
edge  to  the  field  of  learning  of  mathematics 
in  blind  children.  We  are  beginning  to  under¬ 
stand  how  audio-based  interfaces  can  help  to 
develop  mathematics  knowledge  and  skills  in 
blind  learners. 
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Presenter:  Richard  Satava  MD 

ADVANCED  TECHNOLOGIES  FOR  RE¬ 
HABILITATION 

Richard  Satava.  MD 

Telemedicine  and  Advanced  Technology  Research 
Center  (TATRC) 

Numerous  technologies,  such  as  virtual  real¬ 
ity,  brain-machine  devices,  nanotechnology, 
intelligent  prosthetics,  robotics,  and  artificial 
organs  to  name  a  few,  are  opening  new  op¬ 
portunities  for  rehabilitation.  Current  pro¬ 
grams  which  include  exoskeletons  and  intelli¬ 
gent  prostheses  for  rehabilitation  will  focus 
mainly  on  how  the  engineering  sciences  are 
revolutionizing  care  of  the  disabled  and  disad¬ 
vantaged.  Finally  we  are  able  to  bring  new 
technologies  to  restore  function  and  some¬ 
times  realistic  form  for  those  systems  which 
are  lost  or  incapacitated. 


Presenter:  Richard  Satava  MD 

THE  REVOLUTION  IN  HEALTHCARE 
ROBOTICS 

Richard  Satava,  MD 


Telemedicine  and  Advanced  Technology  Re¬ 
search  Center  (TATRC) 

Robotics  has  moved  from  industry  to  health¬ 
care,  creating  a  new  revolution.  The  con¬ 
cepts  behind  this  profound  revolution  are 
discussed,  and  the  various  classes  of  robots 
are  demonstrated,  along  with  their  remark¬ 
able  capabilities  -  from  'robotic  cells'  to  re¬ 
placing  entire  operating  room  teams,  to  rov¬ 
ing  robotic  systems  and  medevacs  on  the 
battlefield,  to  mobile  robots  (unmanned  vehi¬ 
cles)  to  make  rounds  for  the  doctor  and 
nurse.  Finally  speculation  on  the  logical  fu¬ 
ture  that  will  emerge  once  these  robotic  sys¬ 
tems  are  truly  embraced  by  the  medical  pro¬ 
fession 


Presenter:  Ben  Sawyer 

GAMES  FOR  HEALTH:  THERAPY  AND 
BEYOND 

Ben  Sawyer 

Woodrow  Wilson  International  Center  for  Scholars 

In  the  therapy  field  the  use  of  virtual  reality 
(VR)  and  game  technologies  has  been  a 
growing  force  for  some  years  but  the  greater 
health  field  is  just  tuning  in  to  the  possible 
uses  of  games  and  game  technologies.  The 
Games  for  Health  project  is  a  Robert  Wood 
Johnson  Foundation  effort  designed  to  both 
explore  and  identify  relevant  uses  for  game 
technologies  and  the  talents  that  produce 
them  that  could  offer  breakthroughs  in 
healthcare  from  training,  to  health 
messaging,  to  direct  patient  application. 

During  this  presentation  Ben  Sawyer  who 
leads  day-to-day  activities  on  the  games  for 
health  project  will  discuss  the  current  trends 
in  the  games  for  health  field.  With  the  assis¬ 
tance  of  several  other  presenters  the  session 
will  also  highlight  the  logistical  and  industrial 
cultural  issues  which  must  be  hurdled  to  fur¬ 
ther  the  promise  game  technologies  and  ap¬ 
proaches  have  shown. 

Attendees  will  not  only  get  a  sense  of  the 
current  best  practices  but  also  get  a  keen 
sense  of  the  comparative  advantages  (and 
disadvantages)  game-based  approaches  are 
seeking  to  offer  the  health  field. 
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ISLANDS  -  E  -MENTAL  HEALTH  - 
TELEPSYCHIATRY 

Thomas  Senn 

COAT-Basel 

An  overview  of  the  European  project  IS¬ 
LANDS  will  be  provided.  Which  services  are 
feasible  and  which  are  desired,  which  are 
promising?  The  ISLANDS  project  will  be  em¬ 
bedded  in  e-health  and  e-mental  health.  The 
persons  involved,  the  services  provided  and 
the  tools  used  will  then  be  explained.  Different 
questions  will  be  touched:  How  can  informa¬ 
tion  of  counselling  and  therapy  be  formalised 
for  persons  that  are  indirectly  involved 
(informal  carer)  in  the  therapeutic  process? 
What  are  the  process  dynamics  within  an 
online  therapy?  Which  patient  -  therapist  rela¬ 
tionships  can  evolve?  Which  are  the  limits  of 
e-mental  health  service  provisions? 

Within  ISLANDS,  a  cognitive  behavioural  ap¬ 
proach  for  the  therapy  module  was  chosen; 
other  forms  of  therapy  are  imaginable.  Details 
about  our  approaches  will  be  delivered. 

Counselling  is  part  of  a  comprehensive  model 
that  is  founded  on  several  assumptions  about 
human  development:  e.g.,  individuals  achieve 
optimal  functioning  when  they  attain  opera¬ 
tional  mastery  of  fundamental  life  skills  and 
neurosis  and  functional  psychosis  frequently 
result  from  failure  to  develop  certain  life  skills. 

Further  an  overview  of  technological  means  in 
which  ISLANDS  can  be  embedded  in  the  fu¬ 
ture  is  given.  Visions  for  the  future  are  out¬ 
lined. 

Contact: 

Thomas  Senn 
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A  PROGRESS  REPORT  OF  LONG¬ 
TERM  ROBOT  ASSISTED  ACTIVITY  AT 
A  HEALTH  SERVICE  FACILITY  FOR 
THE  AGED 


Kazuyoshi  Wada\  Takanori  Shibata^'^,  To- 
moko  Saito^  Kayoko  Sakamoto^  and  Kazuo 
Tanie^ 

’intelligent  Systems  Research  Institute,  National 
Institute  Advance  Science  and  Technology  (AIST) 
1-1-1  Umezono,  Tsukuba,  Ibaraki  305-8568,  JA¬ 
PAN 

^  National  Institute  Advance  Science  and  Technol¬ 
ogy  (AIST) 

¥eresto,  JST 

Background 

We  have  proposed  Robot-Assisted  Therapy 
and  Activity  since  1996,  and  have  been  de¬ 
veloping  mental  commit  robots  that  provide 
psychological,  physiological,  and  social  ef¬ 
fects  to  human  beings  through  physical  inter- 
action’’®’.  The  appearances  of  these  robots 
look  like  real  animals  such  as  cat  and  seal. 
The  seal  robot,  Paro,  was  developed  espe¬ 
cially  for  therapy.  We  have  applied  seal  ro¬ 
bots  to  therapy  of  children  at  pediatric  hospi¬ 
tals'**  and  to  assisting  activity  of  elderly  peo¬ 
ple  at  a  day  service  center®*'^*.  Recently,  sev¬ 
eral  research  groups  have  tried  robot  as¬ 
sisted  therapy  and  activity.  Dautenhahn  has 
used  mobile  robots  and  robotic  dolls  for  ther¬ 
apy  of  autistic  children^®*.  Besides,  robot- 
assisted  activity  that  uses  commercialized 
animal  type  robots  (such  as  AIBO***,  Ne- 
CoRo,  etc.)  has  been  tried’^*"’'**.  For  exam¬ 
ple,  Yokoyama  used  AIBO  in  a  pediatrics 
ward,  and  observed  the  interaction  between 
children  and  pointed  out  that  the  initial  stimu¬ 
lus  received  from  AIBO  was  strong.  How¬ 
ever,  the  long  term  stability  was  quite  weak, 
compared  with  living  animals’^*.  In  this  pres¬ 
entation,  we  will  explain  the  results  of  the 
robot-assisted  activity  for  elderly  people  at  a 
health  service  facility  for  the  aged  for  more 
than  one  year. 

Method 

In  order  to  investigate  the  effects  of  seal  ro¬ 
bots  to  the  elderly  people,  we  evaluated 
moods  of  elderly  people  by  face  scales*®*  that 
express  person’s  moods  by  illustration  of 
person’s  faces,  questionnaires  of  Geriatric 
Depression  Scales  (GDS)*®*.  Seal  robots 
were  provided  into  the  health  service  facility 
on  two  days  per  a  week  from  Aug.  2003. 

Results 

The  results  of  face  scale  and  GDS  showed 
that  feelings  and  depression  of  elderly  people 
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were  improved  by  interaction  with  the  seal 
robots,  Paro.  Regarding  a  case  study, 

Hanako  (pseudonym),  aged  89,  was  sociable 
and  comparatively  independent.  On  the  first 
day  of  the  interaction  with  Paro,  she  looked  a 
little  nervous  of  the  experiment.  However,  she 
soon  came  to  like  Paro.  She  treated  Paro  like 
her  child  or  grandchild.  Her  face  scale  scores 
after  interaction  were  always  lower  than  be¬ 
fore  interaction  after  the  first  day.  Unfortu¬ 
nately,  she  was  hospitalized  during  Dec.  10  to 
26,  2003.  When  she  met  Paro  for  the  first  time 
after  leaving  hospital,  she  said  to  Paro  "I  was 
lonely,  Paro.  I  wanted  to  see  you  again."  Her 
GDS  score  then  improved.  To  the  present, 
she  has  continued  to  join  the  activity  and  will¬ 
ingly  interacted  with  Paro.  Caregivers  com¬ 
mented  that  interaction  with  Paro  made  the 
people  laugh  and  become  more  active.  For 
example,  their  facial  expression  changed,  sof¬ 
tened,  and  brightened.  In  addition,  Paro  en¬ 
couraged  the  people  to  communicate,  both 
with  each  other  and  caregivers,  by  becoming 
their  common  topic  of  conversation.  Thus,  the 
general  atmosphere  became  brighter. 

Conclusions 

We  have  used  seal  robots,  Paro  in  RAA  for 
elderly  people  at  a  health  service  facility  for 
the  aged  since  August  2003.  The  results 
showed  that  interaction  with  Paro  improved 
their  moods  and  depression,  and  then  the 
effects  showed  up  through  more  than  one 
year.  Consequently,  the  seal  robots,  Paro 
were  effective  for  therapy  at  health  service 
facilities. 
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Journal  of  Psychopharmacology  Bulle¬ 
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Contact: 

Takanori  Shibata 

Intelligent  Systems  Research  Institute 
National  Institute  Advance  Science  and  Tech¬ 
nology  (AIST) 

1-1-1  Umezono,  Tsukuba 
Ibaraki  305-8568,  JAPAN 
Tel.:  +81-29-861-5980 
Fax:  +81-29-861-5992 
Email:  shibata-takanori@aist.go.jp 


Presenter:  Russell  Shilling  PhD 

THE  OFFICE  OF  NAVAL  RESEARCH 
PROGRAM  IN  VR  THERAPY  AND  THE 
USE  OF  "SERIOUS  GAMES"  AND  EN¬ 
TERTAINMENT  TECHNOLOGIES  FOR 
MEDICAL  SIMULATION 

CDR  Russell  Shilling,  Ph.D. 

Office  of  Naval  Research 

Increasingly,  virtual  reality  (VR)  and  videogame 
technologies  are  gamering  more  interest  in  the 
medical  community.  At  the  Office  of  Naval 
Research  (ONR),  programs  have  been  initiated 
which  look  at  treatments  for  combat-related 
Post-Traumatic  Stress  Disorder  (PTSD)  and 
also  evaluate  a  wide  variety  of  VR  medical 
training  platforms.  ONR  is  also  assisting  the 
Defense  Advanced  Research  Project  Agency 
(DARPA)  in  developing  innovative  treatment 
options  for  amputees  using  videogame  tech¬ 
nology.  One  of  the  underlying  philosophies 
that  unify  these  different  programs  is  the  belief 
that  emotion  is  a  crucial  component  for  effec¬ 
tive  simulation  and  that  lessons  learned  from 
the  entertainment  industry  need  to  be  inte¬ 
grated  into  the  development  process.  In  the 
entertainment  industry,  sound  design  is  recog¬ 
nized  as  a  critical  component  in  creating  emo¬ 
tional  content.  Until  recently,  little  effort  was 
made  to  adapt  professional  sound  design  tech¬ 
niques  commonly  used  in  the  entertainment 
industry  for  use  in  simulation.  This  talk  will 
summarize  programs  sponsored  by  ONR  to 
develop  VR  therapy  applications  for  treating 
Sailors,  Soldiers,  and  Marines.  These  PTSD 
programs  will  also  be  discussed  in  the  context 
of  research  and  development  activities  con¬ 
ducted  by  the  author  while  acting  as  the  sound 
designer,  audio  engineer,  and  research  psy¬ 
chologist  working  on  the  popular  videogame, 
America’s  Army.  In  the  process  of  developing 


the  audio  experience  for  the  game,  extensive 
consultations  were  made  with  leading  experts 
and  engineers  from  the  movie  and  videogame 
industry.  As  a  result,  the  game  has  received 
high  reviews  in  the  videogame  industry  for  the 
quality  of  sound  and  has  helped  the  game 
gain  over  4-million  registered  users  worldwide. 
These  same  sound  design  techniques  and 
philosophies  need  to  be  incorporated  in  VR 
therapy  applications  to  help  maximize  the 
emotional  impact  and  increase  the  sense  of 
immersion  in  the  simulation.  Important  les¬ 
sons  were  also  learned  concerning  the  video- 
game  development  process  and  how  it  relates 
to  creating  so-called  “serious  games”  with 
goals  other  than  entertainment.  Current 
trends  in  the  “serious”  use  of  videogame  tech¬ 
nologies  will  be  discussed,  including  the 
“Games  for  Health”  Project  which  held  its  first 
annual  conference  in  Madison,  Wisconsin  in 
2004. 

Contact: 

CDR  Russell  Shilling,  Ph.D. 

Biological  and  Medical  Science  &  Technology 
Division 

Office  of  Naval  Research 

Arlington,  VA  22217 

Email:  Shilling85@alumni.wfu.edu 


Presenter:  Viktoria  Simon  MD 

CLINICAL  EXPERIENCES  IN  USING 
VIRTUAL  REALITY  (VR)  IN  PSYCHO¬ 
THERAPY 

Viktoria  Simon  ^MD,  Vajos  Simon  ^MD, 

Phd, ,  ^Barnabas  Takacs  Phd 

’Semmelweis  University  Budapest,  Department  of 
Psychiatry  and  Psychotherapy 
^DigitalElite,  Los  Angeles  (CA) 

This  year  our  team  in  the  department  of  psy¬ 
chiatry  and  psychotherapy  of  Semmelweis 
University  Budapest  had  the  opportunity  to 
create  and  try  VR  in  treating  phobic  patients 
for  the  first  time  in  Hungary.  The  programs 
that  are  available  for  us  can  be  used  for  the 
therapy  of  travel-phobia  (subway,  driving, 
bus,  airplane),  acrophobia,  claustrophobia. 
Beside  learning  the  conditions  and  tools  we 
make  the  first  steps  needed  for  exposition- 
therapy  together  with  the  patient.  We  teach 
also  relaxation  technics.  For  controlling  the 
relaxation  and  the  process  of  the  exposition 
we  are  monitoring  real-time  the  data  of 
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physiological  parameters  provided  by  special 
biosensors. 

After  this  training  period,  desensitization  takes 
place  in  a  virtual  environment  under  the  con¬ 
trol  of  the  therapist.  In  the  different  environ¬ 
ments  the  therapist  can  change  the  parame¬ 
ters  in  order  to  get  the  phobic  stimuli  stronger 
or  weaker. 

Based  on  our  first  pilot-cases  and  experi¬ 
ences  we  can  enforce  that  desensitization 
with  VR: 

-  makes  the  therapy  practically  easier  to  carry 
out,  also  easier  to  control 

-  can  be  less  dangerous  than  standard  desen¬ 
sitization  (thinking  about  acrophobia  for  exam¬ 
ple) 

-  easier  to  engage  patients  for  therapy 

-  drop  out  is  lower  than  in  standard  therapies 

-  the  stimuli  that  reach  the  patient  can  be  con¬ 
trolled  more  precisely,  the  reactions  given  by 
the  patient  to  the  environment  can  be  meas¬ 
ured  more  precisely. 

Contact; 

Viktoria  Simon  MD 
Semmelweis  University  Budapest 
Department  of  Psychiatry  and  Psychotherapy 
Fax:  +36/1/2100-336 
Email:  simonviktoria@psvch.sote.hu 


Presenter:  Oliver  Stefani 

NEXT  GENERATION  VIRTUAL  REAL¬ 
ITY  SYSTEMS:  FUTURE  CHALLENGES 

Oliver  Stefani,  Dipl.-Ing.  Designer 

COAT-Basel,  Switzerland 

Technologies  that  directly  stimulate  the  brain 
to  simulate  reality  still  are  science  fiction. 
Therefore,  current  VR  technologies  are  bound 
to  the  stimulation  of  the  five  human  physio¬ 
logical  senses.  This  paper  discusses  the  state 
of  the  art  of  current  Virtual  Reality  (VR)  Sys¬ 
tems  and  highlights  their  barriers  and  prob¬ 
lems.  The  focus  will  be  on  stereoscopic  dis¬ 
play  technologies,  CAD-VR  integration  and 
interface  technologies.  We  will  give  insights  in 
the  trends  of  new  VR  technologies  which  we 
identified  within  VIEW  of  the  Future^  and  IN- 
TUITIOt^  and  provide  some  suggestions  to 
improve  current  VR  systems. 


Reference 

^ IST-2000-26089 
^Contract  no.;  507248-2 


Presenter:  Julie  St-Jacques 

CLINICAL  APPLICATIONS  OF  VIR¬ 
TUAL  REALITY  AND  CYBERSICK¬ 
NESS 

Julie  St-Jacques  Ph.D.  candidate  and  Ste¬ 
phana  Bouchard  Ph.D. 

Universite  du  Quebec  en  Outaouais 

For  a  few  years,  virtual  reality  (VR)  has  made 
its  entry  as  a  new  therapeutic  method,  and 
more  and  more  studies  are  documenting  its 
effectiveness,  especially  for  the  treatment  of 
anxiety  disorders.  Cybersickness  is  the  prin¬ 
cipal  counter-indication  with  the  use  of  this 
therapeutic  method  and  are  also  a  major 
concern  for  many  Research  Ethics  Boards. 
Very  few  studies  have  documented  cyber¬ 
sickness  in  clinical  populations  and  research¬ 
ers  as  report  very  low  symptoms.  Moreover, 
our  current  knowledge  on  this  question  is 
often  based  on  studies  conducted  either  with 
very  different  populations  (e.g.,  Navy  pilots  or 
astronauts)  or  with  different  tasks  (e.g., 
search  and  rescue,  complex  motor  tasks). 
These  may  not  generalize  well  to  clinical 
populations  and  therapeutic  tasks.  It  is  there¬ 
fore  important  to  document  cybersickness  in 
a  way  that  is  useful  and  applicable  to  clinical 
situations. 

The  goal  of  this  study  is  to  document  cyber¬ 
sickness  by  collecting  systematic  data  in  chil¬ 
dren  and  adults.  The  sample  of  non  phobic 
participants  consisted  of  23  children  and  34 
adults.  Questionnaire  and  head  motion  data 
were  recorded.  In  order  to  compare  if  the 
cybersickness  reported  by  non  phobics  are 
equivalent  with  those  reported  by  phobic  par¬ 
ticipants,  data  from  adult  phobics  are  also 
used. 

The  results  revealed  few  cybersickness 
symptoms.  When  comparing  children  and 
adults  separately,  there's  no  signifi¬ 
cant  difference  in  cybersickness  reported.  A 
significant  correlation  is  found  between  dis¬ 
comfort  related  to  VR  and  the  amplitude  of 
head  movements  performed  during  the  VR 
immersion.  Finally,  the  relatively  low  level  of 


296 


CyberTherapy  2005  Abstracts 


cybersickness  reported  is  comparable  be¬ 
tween  phobics  and  non  phobics.  The  resuits 
obtained  iliustrate  that  VR  is  safe  when  used 
with  healthy  participants  as  well  as  when  all 
the  precautions  to  ensure  there  comfort  are 
taken. 

Contact: 

Juiie  St-Jacques  Ph.D.  candidate 
Universite  du  Quebec  en  Outaouais 
23  Pharand  Gatineau  J9A  1 K7 
Canada 

Tel:  1-819-771-7761  X  8392 

Fax:  1-819-776-8098 

Email:  iuliestiacques@netscape.net 


Presenter:  Hubert  Sulzenbacher  MD 

TELECOMMUNICATION  IN  PSYCHIA¬ 
TRY:  A  NEEDS  ASSESSMENT  OF  DIF¬ 
FERENT  POTENTIAL  USER  GROUPS 
IN  THE  "ISLANDS"  PROJECT 

Dr.  Hubert  Sulzenbacher 

Department  of  Psychiatry 
Medical  University  Innsbruck,  Austria 

Geography  influences  access  to  mental 
health-care.  There  are  some  regions  in  the 
European  Union  with  particular  geographical 
characteristics,  which  are  responsible  for  their 
being  behind  the  average  socio-economic 
development  in  Europe.  In  such  regions  there 
is  a  lack  of  access  to  modem  health-care  fa¬ 
cilities,  especially  psychiatric  and  psycho¬ 
therapeutic  therapies.  Telecommunication 
technologies,  such  as  the  telephone,  interac¬ 
tive  television,  and  the  Internet,  can  provide 
feasible  tools  for  diagnostic,  counseling  and 
therapy  purposes  to  improve  mental  health¬ 
care  in  such  disadvantaged  areas.  One  has  to 
differentiate  between  several  users'  needs. 
Persons  who  are  suffering  from  a  certain  dis¬ 
order  are  looking  for  different  information, 
seeking  different  forms  and  content  of  help 
than  their  family  members,  partners,  or  friends 
do.  All  of  them  are  involved  in  the  disorder 
and  its  consequences.  So  are  professionals 
who  sometimes  need  advice,  especially  in 
state  of  the  art  treatment  adjusted  to  their 
specific  environment.  "ISLANDS",  a  project 
funded  by  the  Commission  of  the  European 
Union  (QLRT-2001-01637),  aims  to  develop 
services  to  provide  modular,  remote  psychiat¬ 
ric  and  psychotherapeutic  assistance  for  re¬ 
mote  areas.  By  these  means  quality  of  life  of 


the  users,  quality  of  mental  health  care  and 
the  economic  strength  of  the  region  should 
improve  and  overweight  the  costs  of  imple¬ 
mentation  and  service  support.  In  the  project 
a  questionnaire  was  developed  to  identify  the 
needs  of  the  potential  user  groups  of  such  a 
service.  This  questionnaire  was  given  to  pa¬ 
tients,  family  members,  and  therapists  in  five 
European  regions  to  find  relevant  differences 
between  urban  centers  and  remote  rural  re¬ 
gions.  The  results  of  this  survey  showed  a 
dissatisfying  situation  of  mental  health-care 
as  well  as  of  diffusion  of  telecommunication 
technologies  in  remote  regions.  We  found 
little  knowledge  about  and  little  experience 
with  telepsychiatry  and  e-mental  health,  but 
all  potential  user  groups  showed  positive  atti¬ 
tudes  towards  the  psychiatric  use  of  telecom¬ 
munication  media,  and  most  users  were  in¬ 
terested  having  a  telepsychiatric  or  e-mental 
health  service  available. 

Contact: 

Dr.  Hubert  Sulzenbacher 
Department  of  Psychiatry 
Medical  University 
Anichstrasse  35 
Innsbruck  6020,  Austria 
Phone:  004351250423668 
Fax:  004351250423628 
Email:  ullrich.meise@uklibk.ac.at 


Presenter:  loannis  Tarnanas  PhD 

AROUSED  AND  IMMERSED:  INTRO¬ 
DUCING  "VIRTUAL  PAIN"  INTO  AN 
EXTREME  TEAM  TRAINING  EXPERI¬ 
ENCE 

loannis  Tarnanas  PhD 

Kozani  University  of  Applied  Science,  Greece 

Biocybernetics  approaches  for  promoting 
human-system  compatibility  continuously 
adjust  the  challenge  level  of  a  task  to  match 
the  available  resources  of  the  human  opera¬ 
tor  based  on  his/her  psycho  physiological 
state.  This  project  expands  the  scope  of  bio¬ 
cybernetics  to  teams.  The  objective  of  this 
project  was  to  test  the  effectiveness  of  a  se¬ 
lected  training  intervention  enabled  by  VE 
technology.  The  training  intervention  that  was 
selected  was  the  use  of  an  advanced  team 
human-computer  interface  to  enhance  VE 
training  effectiveness.  Specifically,  this  pro¬ 
ject  tested  the  effects  of  one  operator's  psy- 
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cho-physiological  fear  and  anxiety  measures, 
to  another  operator’s  explicit  display  of  control 
"difficulty",  represented  in  a  force  feedback 
Joystick  ("virtual  pain"),  to  handle  cues  in  a 
counter  strike  wargame  simulation.  Prior  re¬ 
search  work  indicated  that  social  psycho 
physiological  compliance  can  predict  team 
performance  in  a  projective  tracking  task 
(Tarnanas,  et  al.,  2003).  Two  instructional  VE 
were  presented.  One  VE  was  using  the  ex¬ 
plicit  biocybernetic  interface  that  challenged 
the  self-efficacy  and  team  decision  making  of 
the  operators  in  a  crisis  situation  scenario,  by 
linking  their  heart  rates  (EGG),  electrodermal 
activity  (EDA)  and  electromyogram  measure¬ 
ments  of  facial  muscle  tensions  (EMG)  with  a 
Force  Feedback  Joystick  of  controlled  "ease" 
of  movements  while  engaged  in  a  joint  task. 
The  second  VE  was  the  same  team  crisis 
situation  environment  as  the  above,  but  with¬ 
out  the  introduction  of  the  advanced  biocyber¬ 
netic  interface.  In  both  the  scenarios  the  op¬ 
erators  had  the  freedom  to  communicate  ver¬ 
bally  while  engaged  in  the  common  task.  Two 
performance  measures  were  computed:  (a) 
RMS  error,  and  (b)  number  of  groundings. 
Analyses  of  RMS  error  scores  indicate  that 
performance  improved  over  the  three  re¬ 
peated  runs  with  each  VE,  as  evidence  of 
specific-channel  learning.  Similarly,  perform¬ 
ance  improved  on  the  first  run  from  Day  1  to 
Day  2,  as  evidence  of  near-transfer  learning. 
However,  the  advanced  biocybernetic  inter¬ 
face  provided  to  the  first  experimental  group, 
provided  an  additional  important  statistical 
performance  improvement  relative  to  the 
"simple"  VE  group  and  the  control  groups. 
The  mapping  between  the  type  of  user- 
operator  state  and  the  type  of  medium  re¬ 
sponse  also  had  an  influence  on  the  level  of 
presence.  This  project  suggests  that  social- 
psychophysiological  measures  and  team  bio¬ 
cybernetics  merit  further  investigation  in  so- 
ciotechnical  systems  that  demand  high  profi¬ 
ciency  and  self-efficacy. 

Contact: 

loannis  Tarnanas  PhD 

Kozani  University  of  Applied  Science 

Thessalonica,  1st  Emm.  Papa  street 

55236  Kozani,  Greece 

Phone:  +302310344831 

Fax:  +302310344810 

Email:  ioannist@auth.gr 


Presenter:  Sharon  Tettegah  PhD 


EXPLORING  ANIMATED  NARRATIVE 
VIGNETTE  TECHNOLOGY  TO  UNDER¬ 
STAND  EMPATHY,  AGGRESSION 
AND  RACIAL  IDEOLOGIES  WITH  MID¬ 
DLE  SCHOOL  YOUTHS 

Sharon  Tettegah,  Ph.D.,  Helen  Neville,  Ph.D. 

University  of  Illinois,  Urbana  Campaign 

Research  status:  Completed  Preliminary 
Clinical  Trials 

Background 

Relational  aggression,  such  as  name  calling 
and  other  epithets,  remain  in  today's  school 
environments.  Despite  the  effort  to  reduce 
relational  aggression  (name-calling,  eye  roll¬ 
ing,  etc.)  in  schools,  such  behaviors  continue 
to  pervade  our  classroom  and  subject  stu¬ 
dents  to  forms  of  unnecessary  victimization. 
This  study  begins  with  a  personal  narrative  of 
an  eleven  year  old  African  American  female, 
which  was  developed  into  a  web-based  ani¬ 
mated  narrative  vignette  (ANV).  An  ANV  is  a 
scenario  that  expresses  a  real  life  experience 
that  mimics  a  situation  (Tettegah,  2002, 

2003,  2004).  As  a  research  and  teaching 
tool,  other  types  of 

vignettes  tend  to  be  positively  received  by 
both  scholars  and  educators  and  the  use  of 
vignettes  has  been  empirically  validated  to 
help  students  achieve  educational  goals  on  a 
variety  of  technical,  social,  and  behavioral 
topics  (Barter  &  Renold,  2000;  Finch,  1987; 
Chau  et  al.,  2001;  Mosquera,  Manstead  & 
Fischer,  2002;  Schoenberg  &  Ravdal,  2000; 
Sleed,  Durrheim,  Kriel,  Soloman,  et  al., 

2002;).  Our  research  leverages  the  storytel¬ 
ling  process  of  creating  an  ANV  by  using 
technology  and  will  leverage  the  deliverables 
of  that  storytelling  process,  the  vignettes,  to 
understand  how  students  perceive,  in  this 
paper,  an  incident  in  the  classroom  (multiple 
types  of  AN  Vs  have  been  created  with  differ¬ 
ent  foci).  This  research  sought  to  answer  the 
following  questions:  Is  ANV  technology  a 
viable  fidelity  to  measure  student  responses 
to  social,  anti-social  and  behavioral  disposi¬ 
tions? 

Method 

Participants.  One  hundred  and  four  middle 
school  students  (N=  55  males,  N  =  45  fe¬ 
males,  4  unidentified)  in  the  inner  city  Chi¬ 
cago  area  participated  in  this  study.  Students 


298 


CyberTherapy  2005  Abstracts 


were  in  the  7th  and  8th  grade  (N=  30,  7th 
graders  and  N  =  70,  8th  graders,  4  unidenti¬ 
fied).  Qualitative  data  analysis  revealed  the 
following  categories: 

Racism  =  acknowledgement  of  racism  or  ra¬ 
cial  prejudice  occurring  in  the  vignette  (0  =  no 
acknowledgement:  1  =  acknowledgement) 
Empathy  =  participant  expressed  empathy 
toward  the  victim  in  the  vignette  (0  =  no  ex¬ 
pressed  empathy;  1  =  expressed  empathy) 
Response  =  how  the  participant  would  re¬ 
spond  to  the  situation.  There  are  three  rows 
representing  physical  aggression,  verbal  re¬ 
sponse,  and  tell  authority.  Physical  Aggres¬ 
sion  (first  row)  -  1  =  response  includes  some 
form  of  physical  aggression  (e.g.,  hit,  beat  up, 
etc.):  0  =  no  physical  aggression  Verbal  Re¬ 
sponse  (second  row)-- 1  =  participant  indi¬ 
cated  verbally  responding  to  Scott  (e.g.,  in¬ 
quired  about  his  intentions  and  thinking,  insult 
him,  etc.);  0  =  no  verbal  response  Tell  Author¬ 
ity  (third  row)-- 1  =  participant  would  respond 
by  telling  an  authority  figure  (e.g.,  teacher, 
principal,  parent);  0  =  no  mention  of  telling 
authority  Affect  =  self  identified  affect  or  per¬ 
sonal  feelings  of  the  participant 
(e.g.,  acknowledging  personally  feeling  mad, 
angry,  hurt,  offended,  etc.)  (0  =  no  acknowl¬ 
edged  affect:  1  =  acknowledged  affect) 

Results 

Data  analysis  suggested  that  students  inter¬ 
preted  the  situation  using  ANV  technology  as 
a  methodological  tool  was  effective.  The  stu¬ 
dent's  were  able  to  take  on  the  perspectives 
of  character(s)  in  the  ANV  and  respond  in 
ways  that  suggest  perspective  taking  and 
problem  solving. 

Novelty/Discussion 

Prior  work  in  this  area  focused  on  pre-service 
teachers  (Tettegah,  2002,  2003,  2004).  Our 
research  provides  the  first  systematic  evalua¬ 
tion  of  the  use  of  an  ANV  technology  as  a 
methodological  tool  with  middle  school  stu¬ 
dents  to  understand  how  students  perceive 
classroom  conflict. 

References 

Barter,  C.  &  Renold,  E.  (2000).  I  wanna  tell 
you  a  story:  exploring  the  application  of  vi¬ 
gnettes  in  qualitative  research  with  children 
and  young  people.  International  Journal  of 
Social  Re  search,  3(4).  307-323 


Finch,  J.  (1987).  The  vignette  technique  in 
survey  research.  Sociology,  21(1).  105-114 
Chau,  J.,  Chang,  A.,  Lee,  I.,  Ip,  W.,  Lee,  D., 

&  Wootton,  Y.  (2001).  Effects  of  using 
videotaped  vignettes  on  enhancing  stu¬ 
dents'  critical  thinking  ability  in  a  baccalau¬ 
reate  nursing  programme.  Journal  of  Ad¬ 
vanced  Nursing,  36(1),  112-119. 

Mosquera,  P.M.,  Manstead,  A.S.R.,  & 

Fischer,  A.H.  (2002)  the  role  of  honour 
concerns  in  emotional  reactions  to  of¬ 
fences.  Cognition  and  Emotion,  16(1).  143- 
163 

Schoenberg,  N.E.,  &  Ravdal,  H.  (2000).  Us¬ 
ing  vignettes  in  awareness  and  attitudinal 
research.  International  Journal  of  Social 
Research  Methodology,  3(1).63-74 
Sleed,  M.,  Durrheim,  K.,  l<riel,A.,  Solo- 
man,V.,  &  Baxter,  V.  (2002).  The  effective¬ 
ness  of  vignette  methodology:  A  compari¬ 
son  of  written  and  video  vignettes  in  eleicit- 
ing  responses  about  date  rape.  South  Afri¬ 
can  Journal  of  Psychology,  32(3). 

Tettegah,  S.  (2002b).  Teachers,  identity, 
psychological  capital,  and  electronically 
mediated  representations  of  cultural  con¬ 
sciousness.  In 

Proceedings  of  the  14th  Ed-Media  World 
Conference  on  Educational  Multimedia, 
Hypermedia  and  Telecommunications  (pp. 
337-340). 

Denver,  CO.  Tettegah,  S.  &  Kien,  G.  (2003, 
April)  Identity,  Human-Computer- 
Interaction,  and  Teacher  Voices.  Paper 
presented  at  the 

Annual  Meeting  of  American  Educational 
Research  Association  Conference,  Chi¬ 
cago,  IL 

Tettegah,  S.  (2004,  August).  Pre-service 
teachers  problem  solving  and  victim  empa¬ 
thy;  What  about  Jamilah?  Poster  presented 
at  the  Annual  Meeting  of  the  American 
Psychological  Association. 

Contact: 

Sharon  Tettegah,  Ph.D. 

University  of  Illinois,  Urbana  Campaign 
383  Education,  MC  708,  1310  S.  Sixth  Street 
Illinois,  United  States 
Phone:  (217)  265-6206 
Fax:  (217)  244-4572 
Email:  stetteqa@ad.ujuc.edu 


Presenter;  Dave  Thomas  PhD 


299 


Annual  Review  of  CyberTherapy  and  Telemedicine 


NIDA'S  VIRTUAL  REALITY  PAIN  RE¬ 
SEARCH  PROGRAM 

Dave  Thomas,  PhD 

National  Institutes  on  Health 
National  Institute  on  Drug  Abuse 

The  National  Institute  on  Drug  Abuse  (NIDA) 
has  an  annual  budget  of  about  a  billion  dol¬ 
lars,  and  is  the  largest  drug  abuse  research 
funding  entity  in  the  world.  NIDA  supports 
basic  drug  abuse  research,  clinical  trials  of 
anti-addiction  medications,  and  research  on 
community  interventions  for  drug  abuse  pre¬ 
vention  and  treatment.  NIDA  also  supports 
many  other  areas  of  research  that  are  related 
to  drug  abuse.  One  example  of  this  is  NIDA’s 
extensive  pain  research  program.  NIDA's 
interest  in  pain  research  is  many  fold,  and 
includes  research  on  why  many  pain  killers 
are  addictive,  how  to  make  analgesics  that 
are  less  addictive,  and  the  development  of 
effective  non-  pharmacological  pain  treat¬ 
ments.  Included  in  this  iater  category  are 
treatments  using  acupuncture,  transcutane¬ 
ous  electrical  nerve  stimulation  (TENS),  and 
virtual  reality  (VR).  NIDA's  VR  and  pain  pro¬ 
gram  is  in  its  infancy,  yet  a  number  of  projects 
have  been  funded  that  have  demonstrated 
efficacy  of  VR  in  the  treatment  of  various 
types  of  pain,  including  dental  pain,  burn  pain, 
and  pain  encountered  during  physical  ther¬ 
apy.  NIDA  is  also  funding  research  for  the 
use  of  VR,  along  with  fMRI  imaging,  to  allow 
people  with  chronic  pain  to  control  the  brain 
areas  responsible  for  their  pain,  thereby  re¬ 
ducing  their  perception  of  chronic  pain.  At 
NIDA,  VR  is  envisioned  as  a  powerful  tool  that 
can  be  used  to  help  many  of  the  millions  of 
people  living  life  with  chronic  pain. 
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Research  Status:  The  paper  describes  re¬ 
search  in  progress. 


Background 

Autistic  Disorder  is  defined  by  social  and 
communicative  impairments  and  restricted, 
narrow  interests.  Impaired  motivation  is  also 
common. 

Several  investigators  have  developed  com¬ 
puter-based  and  virtual  environment  tools  to 
address  various  issues  in  autism(1-5).  Four 
projects  at  CUA  address  the  social  impair¬ 
ment. 

Method/Tools 

Face  Processing:  The  prediction  that  autistic 
face  processing  impairment  involves  gaze 
differences  was  tested  in  an  eye  tracking 
study  using  a  VR  display.  A  replication  in 
progress  uses  a  monitor. 

Early  Intervention:  A  virtual  environment,  with 
kiddie-ride,  monitor  display  and  eye  tracker, 
is  being  tested.  The  goal  is  to  induce  young 
children  with  autism  to  attend  to  faces. 

Social  Navigation:  A  joystick-navigable,  first- 
person-perspective  shopping  mall  is  pre¬ 
sented  on  a  monitor.  To  locate  objects,  the 
user  must  move  either  between  or  around 
social  and  non-social  obstacles.  Data  include 
user  comments  and  path  records. 

Training  in  Social  Conversation:  SIMmersion 
LLC^”^  are  collaborating  with  us  to  develop  a 
social  conversation  module  for  adults  with 
Asperger’s  Disorder.  A  virtual  character  re¬ 
members  the  conversation  and  responds  as 
would  an  actual  interlocutor  to  the  user’s 
speech. 

Results 

Face  Processing:  Preliminary  results  (with 
headset)  confirmed  gaze  differences(6).  The 
replication  is  in  progress.  Early  intervention: 
Reliability-testing  is  being  completed. 

Social  Navigation;  Controls  describe  their 
avatar’s  actions  in  the  first  person,  and  re¬ 
frain  from  walking  between  conversing  char¬ 
acters.  Data  collection  with  persons  with  au¬ 
tism  has  begun.  Training  in  Social  Conversa¬ 
tion:  Development  is  in  progress 

Discussion 

The  projects  described  above  derive  from  the 
hypothesis  that  failure  to  establish  species- 
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typical  face  attention  and  processing  itself 
undermines  cognitive  development(7).  Ac¬ 
cordingly  face  processing  is  targeted  in  young 
children,  and  compensatory  training  is  ad¬ 
dressed  with  older  individuals.  The  challenge 
is  to  make  the  technology  not  only  effective, 
but  adequately  entertaining  to  overcome  au¬ 
tistic  motivational  barriers. 
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AFFECTIVE  IMAGES:  RELIEVING 
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SONS 
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The  Hong  Kong  Polytechnic  University 

Background/Problem 

With  increasing  life  expectancy,  the  inci¬ 
dence  of  chronic  illness  and  chronic  pain  also 
increase.  Chronic  pain  robs  older  people  of 
their  quality  of  life,  limits  functional  mobility 
and  ambulation,  and  this  lead  to  muscle  atro¬ 
phy.  The  older  persons  are  reluctant  to  re¬ 
quest  pain  relief,  attempting  to  endure  pain 
as  'normal'  part  of  ageing.  Chronic  pain  and 
mobility  difficulties  interact  among  older  per¬ 
son;  causing  a  negative  feedback  loop  in 
which  reducing  physical  activity  to  avoid  pain 
usually  ends  up  a  further  increase  in  pain 
when  activity  is  undertaken.  To  this  end,  the 
innovative  non-pharmacological  intervention 
in  pain  management  is  appealing. 

Method/Tool 

Affective  images  and  pictures  were  used  in 
15  elderly  (12  female  and  3  male,  ages 
ranged  60  to  80,  median  age  71)  in  chronic 
pain  and  needed  to  perform  stretching  and 
standing  exercises  in  their  physiotherapy 
session.  The  affective  picture  book  was 
made  according  to  the  preference  of  the 
older  persons.  The  images  included  Chinese 
snack  food  and  dim  sum;  market  areas;  land¬ 
mark  of  the  city;  transportation  system;  flow¬ 
ers  and  natural  scenery  as  well  as  images 
and  pictures  of  previous  famous  movie 
stars.  There  were  about  60  images  and  pic¬ 
tures  in  the  picture  book. 

Pain  scores  were  measured  by  Visual  Ana¬ 
logue  Scale  (VAS)  and  health-related  quality 
of  life  was  measured  by  Medical  Outcomes 
Study  Short  Form  36  (SF  36).  VAS  and  SF- 
36  were  taken  in  week  1  and  in  week  6.  Pain 
scores  were  measured  during  physiotherapy 
session  in  following  weeks. 

Results 

Before  the  intervention,  pain  intensity  was 
found  to  be  severe  (median  pain  scores  was 
50  on  a  100mm  VAS).  There  was  a  signifi¬ 
cant  decreased  of  VAS  from  week  1  to  week 
6  (t  =  3.607;  df  =  14;  p  <0.05).  Also,  a  signifi¬ 
cant  decreased  in  VAS  when  the  older  per¬ 
sons  were  watching  affective  pictures  while 
performing  the  physiotherapy  exercise  in  the 
2nd  and  3rd  week;  but  not  over  the  4th  and 
5th  week.  SF-36  had  been  increased  in 
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week  6  which  indicated  an  increased  in 
health-related  quality  of  life,  despite  not  statis¬ 
tically  significant. 

Discussion/Conclusion 

The  study  demonstrated  a  decreased  in  pain 
perception  and  increased  in  health-related 
quality  of  life  among  the  older  persons  with 
chronic  pain.  Before  the  intervention,  pain 
was  severe  enough  to  hinder  their  activities  of 
daily  living,  decreasing  the  desire  to  partici¬ 
pate  in  exercise  and  social  events. 

Regular  physical  activity  is  important  to 
health.  With  the  use  of  affective  images  and 
pictures,  the  older  persons  tend  to  report  less 
pain;  became  more  cooperated  and  increased 
the  mobility  level  during  and  after  the  physio¬ 
therapy  session.  In  this  regard,  affective  im¬ 
ages  and  pictures  appear  to  be  an  effective 
non-pharmacological  intervention  in  pain 
management  for  the  older  persons,  of  which, 
their  mobility  and  quality  of  life  would  be  en¬ 
hanced. 

Novelty 

There  has  not  been  any  research  and  applica¬ 
tion  of  affective  images  and  pictures  as  an 
adjunct  to  pain  relief  for  older  population.  Our 
study  adds  knowledge  to  existing  pain  relief 
methods. 
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ELS  OF  REAL  DATA  FOR  TRAINING 
THE  VISUALLY  IMPAIRED 
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Informatics  and  Telematics  Institute/CERTH 
Research  Status:  Completed. 

This  paper  presents  a  completed  novel  frame¬ 


work  for  haptic  interaction  with  3D  virtual 
scenes  automatically  generated  from  a  video 
captured  by  a  single  camera.  The  system  is 
used  for  training  visually  impaired  users. 

Background/Problem 

The  combination  of  two  different  and  comple¬ 
mentary  modalities  like  haptics  and  video  is  a 
very  challenging  research  area.  The  potential 
benefits  of  such  a  combination  for  the  visu¬ 
ally  impaired  people  are  very  high  [1],  since 
they  will  be  able  to  navigate  in  unknown  for 
them  places  without  risking  injuries  etc.,  as 
will  be  described  in  the  sequel. 

Method/Tools 

Initially,  bi-directional  2D  motion  estimation  is 
performed  on  the  video.  Rigid  3D  motion  and 
structure  is  recovered  using  an  object  based 
reconstruction  approach,  which  makes  use  of 
extended  Kalman  filtering  (EKF),  utilizing 
also  a  basic  knowledge  of  the  scene.  The 
resulting  3D  structure  data  are  used  to  gen¬ 
erate  the  3D  model  of  the  processed  scene. 
This  is  accomplished  either  by  generating  a 
single  mesh  of  the  scene  utilizing  the  gener¬ 
ated  depth  maps,  or  by  applying  modeling  for 
the  objects  of  the  scene  using  analytical  im¬ 
plicit  functions  and  non-linear  least  squares 
minimization.  The  generated  model  is  used  in 
order  to  provide  haptic  access  to  the  proc¬ 
essed  scene  for  the  blind  using  haptic  de¬ 
vices  like  the  PHANToM  and  the  Cyber- 
Grasp. 

Results 

In  order  to  evaluate  the  proposed  methods, 
experiments  were  performed  on  3D  real 
maps  of  buildings  as  well  as  indoor  maps. 
The  camera  captures  the  map  and  the  sys¬ 
tem  converts  the  monoscopic  video  into  a  3D 
model.  Finally,  the  blind  user  navigates 
through  the  map  using  the  haptic  devices  [2]. 
The  experiments  demonstrated  the  efficiency 
of  the  proposed  methods.  The  system  per¬ 
forms  in  real  time  and  if  there  exists  knowl¬ 
edge  about  the  captured  scene,  like  the  case 
with  the  3D  maps,  the  scene  can  be  approxi¬ 
mated  using  implicit  functions  and  computa¬ 
tionally  intensive  processes  like  collision  de¬ 
tection  are  performed  20-30  times  faster, 
when  compared  to  standard  mesh  based 
methods. 

Usability  Studies 
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The  proposed  haptic  interaction  system  has 
been  also  evaluated  in  tests  with  users  of  the 
Blind  Association  in  Thessaloniki,  Greece. 

The  system  evaluation  results  have  shown 
that  users  consider  it  very  innovative  and  sat¬ 
isfactory  in  terms  of  providing  realistic  and 
smooth  force  feedback.  The  percentage  of  the 
satisfied  users  was  reported  to  be  more  than 
96\%.  Analysis  of  variance  (ANOVA)  test 
were  also  performed  and  illustrated  interest¬ 
ing  statistical  results  (i.e.  the  gender  does  not 
affect  the  performance  results  of  the  user, 
there  exists  a  difference  in  performance  for 
users  which  were  born  blind).  These  tests  are 
not  presented  in  detail  due  to  space  restric¬ 
tions. 

Conclusion 

In  terms  of  novelty  the  proposed  framework 
combines  video  with  haptics  in  a  real  time 
framework  for  the  generation  of  virtual 
scenes,  which  are  then  used  to  train  visually 
impaired  people,  on  how  to  navigate  in  spe¬ 
cific  areas.  Moreover  it  extends  structure  from 
motion  (SfM)  methods  and  combines  them 
with  surface  approximation  schemes  using 
implicit  surfaces,  thus  resulting  in  an  efficient, 
realistic  and  user-friendly  system. 
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The  proposed  paper  describes  a  force  feed¬ 
back  control  system  for  the  performance  of 
remote  ultrasound  examination. 

Background/Problem 

During  the  last  few  years,  there  have  been  a 
lot  of  efforts  to  use  and  integrate  virtual  envi¬ 
ronments  in  medical  applications.  Such 
works  have  been  developed  in  order  to  im¬ 
prove  the  performance  of  specific  medical 
applications.  In  our  case,  a  3D  virtual  envi¬ 
ronment  is  developed  to  provide  the  expert 
with  a  realistic  simulation  of  remote  ultra¬ 
sound  examination.  The  proposed  paper  de¬ 
scribes  the  force  feedback  [1]  control  aspects 
of  a  remote  ultrasound  examination  system. 
The  haptic  interface  is  used  by  an  ultrasound 
specialist  in  order  to  control  a  robotic  system 
and  perform  a  remote  ultrasound  examina¬ 
tion. 

Method/Tools 

The  PHANToMT  [2]  fictive  probe  can  provide 
sufficient  data  for  the  control  of  the  robot, 
which  is  located  at  the  remote  site,  and  offers 
the  capability  to  use  force  feedback  to  assist 
the  expert  performing  the  echography  exami¬ 
nation.  In  the  proposed  virtual  reality  3D  envi¬ 
ronment  a  menu  provides  the  user/expert 
with  the  ability  to  select  between  a  variety  of 
actions  and  functionalities.  Specifically,  the 
user  can  view  a  representation  of  the  work¬ 
space.  The  user  in  the  expert  site  can  simply 
import  appropriate  3D  geometries  from 
VRML  files.  The  expert  can  select  the  exami¬ 
nation  working  area  using  the  PHANToMT 
Stylus  and  start/stop  the  examination  proc¬ 
ess.  When  the  expert  selects  the  examina¬ 
tion  area  an  automatic  scaling  occurs  so  that 
the  active  PHANToMT  workspace  size  can 
access  all  the  workspace  of  the  slave  robot, 
using  a  uniform  scale  factor.  When  the  ex¬ 
pert  starts  the  examination,  the  force  feed¬ 
back  is  applied  to  the  expert  via  PHANToMT 
depending  on  the  force  measured  by  the 
slave  robot  located  at  the  remote  station.  The 
3D  model  can  be  also  used  as  an  alternative 
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source  for  providing  this  force  feedback  input 
to  the  PHANToMT  device,  in  cases  of  weak 
communication  links.  For  safety  reasons,  if 
the  PHANToMT  probe  penetrates  the  3D 
graphical  model  over  a  threshold  value,  an 
alarm  sound  is  activated  to  warn  the  expert 
for  possible  communication  failure.  The  use  of 
a  master  probe  and  slave  robot  brings  up  the 
problem  that  they  have  different  workspaces. 
Using  scaling  can  solve  the  problem  of  differ¬ 
ent  workspaces.  If  scaling  is  applied  only  to 
the  positions  and  the  forces  felt  by  the  human 
remain  the  same,  as  measured  at  the  remote  . 
site,  then  the  appearing  stiffness  of  the  envi¬ 
ronment  changes  due  to  the  scaling  factor  "s". 
In  order  to  solve  this  problem  scaling  is  also 
performed  to  the  force  feedback. 

Results 

The  system  was  integrated  with  a  communi¬ 
cation  system  for  the  transmission  of  ultra¬ 
sound  images  and  video  and  it  was  tested. 
The  results  have  shown  that  the  doctors  ap¬ 
preciated  the  haptic  feedback  when  the  com¬ 
munication  bandwidth  was  good  enough  to 
provide  feedback  to  the  user  without  noticable 
delay. 

Conclusion 

In  this  paper,  we  presented  a  new  3D  virtual 
reality  environment  with  force  feedback  con¬ 
trol  and  inspection  of  ultrasound  examination. 
The  system  was  successfully  integrated  and 
tested  to  a  Tele-Echography  environment. 
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This  is  a  time  to  venture  for  hypnotherapy.  All 
things  change  in  a  human  being.  The  strange 
link  between  us  and  machines  is  a  reality 
today.  Milton  Erickson  gives  us  a  lesson  for 
the  coming  years.  His  mysterious  work,  only 
in  this  millennium  appears  as  a  secret  code 
that  will  be  revealed  to  the  new  genera¬ 
tion.  With  the  case  of  “February  Man"  it  is 
possible  for  us  to  give  a  new  interpretation  to 
it.  This  is  a  new  era  for  hypnosis.  Do  we 
stand  by?  Our  answer  is  in  this  work  the 
PROJECT  IR  in  which  for  the  first  time  hyp¬ 
nosis  is  utilized  with  virtual  reality.  Therapist 
and  patients  together  live  in  a  virtual  pro¬ 
gram.  In  this  project  synthesis  there  are  two 
possible  programs.  One  is  the  development 
of  relaxation  with  a  standard  program  of  vir¬ 
tual  reality  for  all  people  that  want  to  remove 
themselves  out  of  their  daily  life.  This  pro¬ 
gram  is  made  for  a  psychologist’s  clinic  or  in 
the  future  when  it  will  be  available  as  a  video- 
game  in  personal  homes.  Two  is  a  specific 
virtual  reality  program  for  grave  pathology 
such  as  phobias,  depression,  psychosis,  and 
schizophrenia. 

This  virtual  reality  program  is  made  after 
studying  a  deep  history  of  patience.  Each 
program  has  the  same  key  of  opening  and 
closure  with  particularly  pictures. 
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human  emotions  has  been  one  of  the  areas  of 
interest  within  the  scientific  community.  Most 
of  the  studies  pubiished  so  far  are  based  on  a 
combination  of  two  techniques:  advanced  im¬ 
age-processing  and  codification  of  expres¬ 
sions.  This  second  approach  is  based  on  the 
theories  that  postulate  the  existence  of  basic 
and  universal  emotions  and  establish  a  rela¬ 
tionship  between  these  basic  emotions  and 
certain  facial  expressions.  Up  to  now,  a  good 
strategy  for  detecting  a  subject  emotion  has 
been  the  analysis  of  facial  images  to  deter¬ 
mine  the  expressions  shown  by  the  individual, 
and  the  subsequent  codification  of  these  ex¬ 
pressions  in  terms  of  basic  emotions.  Never¬ 
theless,  this  abstract  brings  up  an  innovative 
method  for  the  detection  and  recognition  of 
human  emotions:  the  processing  of  biomedi¬ 
cal  signals. 

This  novel  technique  has  been  the  origin  of 
the  AUBADE  project  (partially  funded  by  the 
EC  1ST  programme)  that  aims  to  develop  an 
intelligent,  multi-sensor  and  wearable  system 
for  the  assessment  of  the  emotional  state  of 
humans  under  special  or  extreme  conditions. 
The  emotions  are  detected  after  the  process¬ 
ing  of  electromyogram  (EMG)  measurements 
obtained  from  the  face  of  the  person  to  be 
assessed.  Additionally  other  biomedical  sig¬ 
nals  are  considered:  hearth  rate  variability, 
skin  conductivity  and  respiration  rate.  This 
approach  means  an  alternative  to  the  tradi¬ 
tional  image-processing  techniques  and 
makes  possible  the  application  of  the  system 
in  a  wider  number  of  situations,  especially  in 
those  were  it  is  not  achievable  to  have  facial 
images  available.  The  project,  now  in  pro¬ 
gress,  involves  the  utilization  of  innovative 
technologies,  such  as  wearable  devices,  bio¬ 
sensors,  data  fusion,  medical  decision  sup¬ 
port  systems,  3-D  animation  and  telecommu¬ 
nications. 

The  result  of  the  project  will  be  a  modular  and 
multifunctional  system  to  be  applicable  in  di¬ 
verse  areas.  In  the  health  sector,  it  is  ex¬ 
pected  that  it  will  contribute  to  improve  the 
diagnosis  of  neurological  diseases  as  well  as 
to  get  a  better  comprehension  of  the  psycho¬ 
logical  status  of  patients  through  the  emotion 
detection.  It  will  be  used  to  monitor  people 
under  high  levels  of  stress  or  special  condi¬ 
tions,  making  possible  to  assess  the  effects 
that  these  conditions  produce  to  their  emo¬ 
tional  state.  The  system  will  also  establish 
relations  between  facial  expressions  and  cer¬ 
tain  diseases,  helping  health  professionals  to 


gain  knowledge  in  this  area  and  increasing 
the  efficiency  in  diagnosis. 

AUBADE  project  will  develop  the  next  gen¬ 
eration  of  the  remote  emotion's  monitoring 
systems,  providing  health  professionals  with 
an  innovative  tool  that  will  lead  them  to  a 
deep  study,  analysis,  understanding,  and 
comprehension  of  human  emotions. 
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ADAPTIVE  DISPLAYS:  CAN  WE  PRE¬ 
DICT  PERFORMANCE? 
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Research  Status:  In  Progress. 

Background 

Adaptive  displays  are  often  envisioned  as 
systems  that  respond  to  state  changes  in  an 
operator.  To  date,  physioiogic  measures  are 
typically  averaged  over  a  window  of  some 
duration,  often  several  minutes,  to  develop 
statistical  certainty  on  the  onset  of  a  state 
change.  We  are  interested  in  determining  the 
possibility  of  using  physiologic  measures  to 
more  directly  predict  the  onset  of  perform¬ 
ance  changes  during  tasks.  To  do  so,  we  are 
examining  oculomotor  changes  as  they  pre¬ 
cede  performance  on  a  dual  auditory  and 
visual  tracking  task.  In  line  with  previous 
studies  in  this  area  (1,2),  we  anticipate  that  a 
number  of  oculomotor  behaviors  will  change 
with  performance  and  that  an  aggregate 
measure  will  have  higher  predictive  power 
that  an  individual  measure.  Our  aspiration  is 
to  be  able  to  monitor  oculomotor  activity, 
perhaps  combined  with  other  behavior  meas¬ 
ures,  to  detect  work  overload  conditions 
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where  errors  might  occur.  Such  approaches 
would  be  applicable  to  situations  such  as  the 
common  driving-while-talking-on-the-phone 
and  could  be  used  to  warn  drivers  that  their 
attention  is  overloaded. 

Method/Tools 

Eye  movements  and  head  were  recorded 
EyeLink  II  from  SR  Research.  The  interactive 
STIO  fixed-base  driving  simulator  developed 
by  Systems  Technology,  Inc.  with  custom 
visual  scenes  were  used  for  a  curving  moun¬ 
tain-driving  scenario.  Participants  were  asked 
to  maintain  lane  position  between  two  moving 
vehicles  a  fixed  distance  apart.  The  secon¬ 
dary  task  was  a  version  of  the  Paced  Auditory 
Serial  Addition  Test  (PASAT).  Subjects  hear  a 
series  of  numbers,  mentally  add  a  number  to 
the  previous  number  and  verbally  state  the 
sum.  Difficulty  is  varied  as  a  function  of  num¬ 
ber  presentation  speed.  A  typical  number 
presentation  rate  of  1 .8  seconds  results  in 
approximately  60%  correct  responses.  The 
subjects  were  instructed  to  perform  three  ten- 
minute  block  trials  of  driving,  auditory,  and  the 
dual  tasks.  Eye  movement  behaviour  charac¬ 
teristics  including  those  of  blinks,  saccades, 
vergence  eye  movements  and  pupil  diameter 
are  all  being  examined  for  their  statistical  rela¬ 
tion  to  the  correct  or  incorrect  responses  on 
the  audio  task  and  to  the  variability  on  the 
driving  performance  task. 

Results 

Preliminary  data  analysis  has  progressed  to 
description  of  overall  comparisons  of  perform¬ 
ance  on  the  combined  task  compared  to  the 
individual  driving  or  audio  task.  Comparison  of 
performance  on  the  auditory  task  alone  ver¬ 
sus  with  driving  showed  that  there  was  small, 
non-significant  decrease  in  performance  with 
the  dual-task.  Blink  durations  increased  in  the 
combined  task  versus  the  driving  task  alone, 
but  actually  substantially  increased  during  the 
auditory  task  alone.  Further,  the  variability  of 
blink  duration  showed  a  similar  pattern.  Fur¬ 
ther  analyses  are  ongoing. 

Conclusion/Discussion 

We  hope  that  this  study  will  provoke  discus¬ 
sion  as  to  the  feasibility  of  using  physiologic 
measures  as  direct  predictors  of  performance 
as  opposed  to  general  indicators  of  operator 
state.  Improved  prediction  could  lead  to  ad¬ 
vanced  adaptive  interfaces  and  direct  im¬ 


provements  in  performance.  If  predictive 
measures  can  be  described  for  our  particular 
tracking/audio  task,  it  will  be  important  to  de¬ 
termine  how  generalizable  our  model  of  cog¬ 
nitive  overload  and  our  metrics  can  be  for 
other  high  work-load  conditions. 
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The  use  of  Virtual  Reality  in  therapy  is  not  a 
novelty  [1]  and  we  expect  that,  combined  with 
different  relaxation  techniques,  it  can  enhance 
the  relaxation  by  visually  presenting,  and 
thereby  enhancing,  key  images  for  facilitating 
relaxation  and  acceptance,  and  enabling  par- 
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tici  pants  to  practice,  and  hence  master,  relaxa¬ 
tion  and  acceptance  techniques  in  a  more  real¬ 
istic  context. 

The  aim  of  the  study  is  to  explore  whether  the 
effects  on  mood,  specifically  on  anxiety  and 
positive  and  negative  states,  of  a  therapeutic 
narrative  enhance  through  its  presentation 
within  a  virtual  environment  [2]  in  an  immer¬ 
sive  condition. 

To  accomplish  this  goal  we  use  the  Relaxa¬ 
tion  Island  in  which  we  apply  different  relaxa¬ 
tion  techniques,  following  a  specific  protocol 
that  is  composed  by  two  sessions.  Three 
phases  compose  each  session:  the  first  is 
based  on  immersive  navigation  in  the  day,  the 
second  on  imagination  and  the  third  on  im¬ 
mersive  navigation  in  the  night. 

The  techniques  aim  to  reduce  anxiety  linked 
to  active  negative  thinking  through  cognitive 
control  and  to  achieve  progressive  muscular 
relaxation  following  the  approach  proposed  by 
Jacobson  [3]  and  the  breathing  techniques. 

To  test  the  enhance  of  the  effects  we  com¬ 
pare  this  procedure  with  an  usually  video  pro¬ 
cedure,  a  new  age  DVD  with  relaxing  music, 
where  participants  watch  a  video  and  have 
the  freedom  to  imagine  any  sensory  element 
required.  Some  of  the  questionnaires  used 
are  Positive  and  Negative  Affect  Schedule 
(PANAS),  to  measure  the  positive  and  nega¬ 
tive  affects  [4],  and  State  Trait  Anxiety  Inven¬ 
tory  (STAI),  to  measure  the  level  of  anxiety[5]. 
Participants,  University  students,  were  30 
subjects  ranging  from  21  to  24  years  old,  split 
up  in  two  groups.  The  study  is  in  progress  and 
the  results  will  be  ready  in  march,  therefore 
preliminary  outcomes  will  be  presented  during 
the  conference. 
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Research  Status:  In  Progress 

Background 

Patients  who  have  had  a  stroke  constitute  a 
large  population  with  significant  needs  for 
rehabilitation.  To  date,  even  after  long,  inten¬ 
sive  and  costly  rehabilitation,  these  patients 
still  suffer  from  many  impairments  leading  to 
severe  restrictions  of  participation  in  every 
day  life.  In  addition,  there  is  insufficient  train¬ 
ing  of  instrumental  activities  of  daily  living 
(lADL)  functioning  during  the  rehabilitation, 
since  it  is  both  time  consuming  and  techni¬ 
cally  difficult  to  implement.  There  is  a  great 
advantage  to  remediate  motor  and  executive 
function  deficits  via  practicing  tasks  that  are 
functional,  meaningful  to  the  patient,  and 
which  can  be  repeated  over  and  over  again 
in  a  safe  environment.  A  Virtual  Mall  (VMall) 
has  been  designed  to  be  used  for  the  treat- 
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merit  of  patients  following  a  stroke  who  have 
motor  and/or  executive  functions  deficits  (1). 

Aims 

1)  To  describe  the  development  of  a  func¬ 
tional  virtual  environment,  the  Virtual  Mall.  2) 

To  present  data  from  a  pilot  study  of  stroke 
patients  which  assessed  the  suitability  of  the 
VMall  for  this  population,  and  3)  To  present 
the  data  of  an  initial  clinical  trial  which  exam¬ 
ines  the  effectiveness  of  the  VMall  environ¬ 
ment  for  the  assessment  and  treatment  of 
executive  functions  deficits  and  motor  deficits. 

Method 

Participants:  For  testing  the  suitability  of  the 
VMall,  six  patients,  aged  50-74  years,  who 
had  sustained  a  stroke  more  than  two  years 
prior  to  the  study  were  pilot-tested.  For  the 
pilot  clinical  trial  stroke  patients,  aged  50-85 
years,  who  had  sustained  a  stroke  at  least 
three  months  prior  to  the  study  will  be  re¬ 
cruited.  The  patients  all  suffer  either  from  a 
considerable  motor  impairment  of  their  upper 
extremity  (assessed  using  the  Fugl-Meyer 
Motor  Assessment)  and/or  from  deficits  in 
their  executive  functions  (assessed  using  two 
subtests  from  the  Behavioral  Assessment  of 
the  Dysexecutive  syndrome  (BADS)  (2)).  In¬ 
struments:  The  VMall  (to  date  one  large  store 
with  multiple  aisles),  has  been  implemented 
via  VividGroup's  (www.vividgroup.com)  video 
capture  Gesture  Xtreme  system  (GX).  The 
performance  of  the  task  provides  multiple  op¬ 
portunities  to  make  decisions,  plan  strategies 
and  multitask,  all  in  a  fairly  intuitive  manner. 
Shopping  for  grocery  products  is  done  by  vir¬ 
tually  touching  the  different  items  located  on 
shelves  in  different  aisles.  Output  measures 
record  the  user’s  performance  with  in  the 
VMall  and  the  kinematics  of  the  patient’s 
movements.  Procedure:  A  single  case  study 
design  (ABA)  will  be  used  to  assess  the  effec¬ 
tiveness  of  treatment  using  the  VMall. 

Initial  Results/Conclusion 

Initial  performance  data  and  feedback  from 
the  patients  suggest  that  the  VMall  provides 
an  interesting  and  motivating  task.  Detailed 
results  of  the  initial  clinical  trial  will  be  pre¬ 
sented  and  further  development  of  the  VMall 
paradigm  will  be  discussed. 

Novelty 


In  comparison  to  functional  virtual  environ¬ 
ments  that  are  presented  on  desktop  VR  sys¬ 
tems  or  those  which  use  a  Head-mounted 
Display,  the  use  of  a  video  capture  platform 
for  the  development  of  a  virtual  functional 
environment  is  advantageous  since  it  encour¬ 
ages  patients  to  actively  move  their  affected 
upper  extremity  in  a  relatively  functional  man¬ 
ner  while  planning,  initiating,  and  problem 
solving  when  engaged  in  the  shopping  task. 
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Background/Problem 

Virtual  Reality  (VR)  is  an  established  neuro¬ 
scientific  instrument.  Several  approaches  to 
develop  software  systems  for  particular 
neuropsychological  experiments  exist,  like 
navigation^  or  driving  assessment.^  An  over¬ 
view  is  given.^  In  other  studies  game  engines 
have  been  used.'*  These  systems  are  in  gen¬ 
eral  limited  to  a  specific  field  of  application. 
When  experimental  paradigms  are  changed, 
the  software  has  to  be  re-designed.  In  neuro¬ 
psychological  studies  exact  runtime  behav¬ 
iour  of  the  software  has  to  be  guaranteed 
and  measured  as  well  as  the  responses  of 
participants  within  millisecond  accuracy,  fea¬ 
tures  that  are  not  provided  in  most  existing 
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approaches.  Finally,  some  of  the  systems 
only  allow  execution  on  desktop-based  sys¬ 
tems,  lacking  immersive,  stereoscopic  visuali¬ 
zation. 

In  order  to  use  Virtual  Reality  in  the  broad 
field  of  neuropsychological  research,  and  to 
meet  diverse  needs  and  requirements  of 
neuropsychological  researchers,  we  have  de¬ 
veloped  the  NeuroMan  system,  a  comprehen¬ 
sive  software  framework.  NeuroMan  enables 
the  user  to  describe  experimental  set-ups  and 
their  runtime  behaviour  in  a  versatile  and  easy 
way  and  to  monitor  the  responses  of  partici¬ 
pants.  With  our  approach  we  have  handled 
the  challenges  of  diversity  of  experimental 
set-ups,  chronological  logging,  and  heteroge¬ 
neous  virtual  environments. 

Method/Tools 

The  design  of  NeuroMan  is  based  on  the  prin¬ 
ciple,  that  a  description  of  the  experimental 
set-up  and  its  chronological  performance 
should  be  separated  from  the  implementation 
of  VR-methods  and  their  particular  implemen¬ 
tation  on  VR-hardware.  In  NeuroMan  experi¬ 
mental  set-ups  are  defined  by  scripts.®  The 
system  generates  a  virtual  experiment  with 
the  desired  application  flow,  i.e.,  a  sequence 
of  sessions,  blocks,  trials  and  multimodal 
stimuli,  which  may  be  influenced  by  specified 
user  interactions  or  system  events.  The  re¬ 
sulting  experiments  can  be  executed  on  a 
variety  of  VR-hardware,  making  use  of  the 
VR-toolkit  ViSTA,  and  its  multimedia  exten¬ 
sions.®  Main  features  of  NeuroMan  are  the 
possibility  to  log  the  overall  system  behaviour 
with  specific  timing  characteristics^  and  to 
handle  large  environmental  models  as  visual 
stimuli.  Integration  of  acoustics,  human  ava¬ 
tars,  and  fMRI  scanner  support  are  being  de¬ 
veloped  in  current  projects. 

Results 

We  have  conducted  several  neuropsychologi¬ 
cal  experiments  with  NeuroMan,  e.g.,  re¬ 
search  on  mental  representations  of  num¬ 
bers,®  on  attentional  asymmetries  after  sleep 
deprivation®  and  for  the  assessment  and  treat¬ 
ment  of  disorders  of  spatial  representations. 

In  a  further  study  we  investigated  distance 
estimation  in  VR.^°  Studies  concerning  the 
influence  of  language  properties  on  object 
manipulation  and  the  modulatory  effects  of 
attention  on  pointing  and  grasping  move¬ 
ments  are  currently  underway.  The  experi¬ 


ments  described  have  been  carried  out  on 
table-like  displays,  on  a  HoloBench™,  in  a  5- 
sided  CAVE-like  environment  and  with  a 
head-mounted  display.  Electro-magnetic  and 
optical  tracking  systems  have  been  used. 

Conclusions 

NeuroMan  has  been  used  successfully  in 
several  neuropsychological  studies.  The 
script  language  with  its  underlying  experi¬ 
mental  model  is  capable  of  describing  vari¬ 
ous  experimental  set-ups 

Novelty/Discussion 

Even  though  several  approaches  to  the  de¬ 
velopment  of  software  systems  for  neuropsy¬ 
chological  experiments  have  been  utilized,  a 
comprehensive  software  framework  for  VR- 
based  experiments  has  not  been  established 
yet.  NeuroMan  is  capable  of  handling  a  vari¬ 
ety  of  experiments  on  a  comprehensive  and 
homogeneous  basis. 
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PRESENCE-  SETTING  THE  SCENE 
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Much  attention  has  been  focused  on  feeling  a 
sense  of  presence  in  a  virtual  environment 
(VE).  The  potential  importance  of  presence  is 
based  on  an  assumption  that  increasing  the 
sense  of  presence  experienced  in  a  VE  leads 
to  improved  task  performance. 

The  goal  of  this  informal  talk  is  to  set  the  scene 
for  the  following  presentations  by  introducing 
the  presence  construct.  It  will  summarize  the 
results  of  over  120  studies  that  have  been  con¬ 
ducted  by  various  researchers  over  the  last 
decade.  Critical  outstanding  research  ques¬ 
tions  will  be  identified,  along  with  some  recom¬ 
mendations  for  future  presence  research. 
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VR  FOR  BLOOD-INJECTION-INJURY 
PHOBIA 

Brenda  K.  Wiederhold’'^,  Megan  Mendoza’, 
Tadashi  Nakatani’, 

Alex  H.  Bullinger^,  Mark  D.  Wiederhold’ 

’  Virtual  Reality  Medical  Center,  San  Diego, 
California 

^Interactive  Media  Institute,  San  Diego,  California 
^COAT-Basel/UPKBS,  University  of  Basel, 
Switzerland,  Department  of  Psychiatry 

Virtual  reality  (VR)  exposure  therapy  has 
been  shown  to  be  successful  in  treating 
many  types  of  specific  phobias  which  are 
mostly  visual  in  nature.  However,  limited 
research  has  been  completed  on  the  use  of 
VR  therapy  for  Blood  Injection-Injury  (BN) 
phobias,  one  of  the  subtypes  of  Specific  Pho¬ 
bia  listed  in  the  DSM-IV  TR.  Since  Bll  pho¬ 
bia  may  operate  by  some  tactile  component, 
it  may  respond  differently  to  VR  therapy  com¬ 
pared  to  other  categories  of  specific  phobias 
that  are  largely  visually  activated.  This  paper 
discusses  initial  development  and  results 
from  a  study  on  both  subjective  and  objective 
arousal  elicited  by  a  prototype  virtual  world 
which  has  been  developed  to  treat  those  with 
Bll  phobia.  The  present  study  evaluated  the 
responses  of  20  healthy,  non-phobic  male 
and  female  participants  to  VR  blood  and  in¬ 
jection  stimuli.  Initial  results  are  positive  and 
show  that  the  VR  world  delivers  appropriate 
cues  to  elicit  physiological  and  self-reported 
arousal  when  exposed  to  the  injection  sce¬ 
narios.  Correlations  between  self-reported 
anxiety  and  physiological  arousal  confirm 
that  individuals  experiencing  greater  symp¬ 
toms  of  fear  in  conditions  involving  blood  or 
injections  will  exhibit  more  intense  arousal 
from  the  virtual  stimuli  than  those  who  experi¬ 
ence  reduced  symptoms.  Findings  suggest 
that  the  virtual  world  is  an  effective  method  of 
cue  exposure  for  individuals  who  experience 
anxiety  in  situations  related  to  blood  and  in¬ 
jections.  Future  research  on  the  use  of  VR 
exposure  therapy  in  the  treatment  of  Bll  pho¬ 
bia  is  warranted. 
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TOWARDS  A  COGNITIVE  AND  SENSO¬ 
RIAL  MODEL  OF  PRESENCE:  A 
STUDY  IN  DIFFERENT  VIRTUAL  SEN¬ 
SORY  CONDITIONS 

Feryel  ZnaTdi,  MA,  Olivier  Warusfel,  Roland 
Jouvent,  IsabelleViaud-Delmon 

CNRS  UMR  7593 

Presence  is  the  experience  a  person  has, 
when  in  a  virtual  environment  (VE),  of  'being 
there.'  Presence  is  an  important  concept  to 
assess  the  effectiveness  of  a  virtual  environ¬ 
ment  in  clinical  applications.  We  designed  an 
experiment  in  which  the  importance  of  differ¬ 
ent  factors  that  might  contribute  to  the  feeling 
of  presence  could  be  evaluated.  Three  kinds 
of  VEs  in  which  different  sensory  information 
were  provided  to  the  subjects  were  tested  and 
compared.  Subjects  were  equipped  with  a 
head-mounted  display  coupled  with  an  elec¬ 
tromagnetic  sensor  system  and  immersed  in  a 
virtual  town  in  which  they  could  move  forward 
by  pressing  a  mouse  button.  Subjects  had  to 
turn  on  their  own  vertical  axis  in  order  to 
change  the  direction  of  heading  in  the  virtual 
town.  Their  task  was  to  locate  different  land¬ 
marks  and  become  familiar  with  the  town.  Af¬ 
ter  this  virtual  navigation,  they  had  to  fill  in  a 
presence  scale,  a  state  anxiety  scale,  a  cyber 
sickness  symptoms  scale,  and  complete  sev¬ 
eral  spatial  memory  tests  related  to  their  ex¬ 
perience.  Subjects  had  to  perform  this  naviga¬ 
tion  according  to  three  conditions  (Vis,  Avis, 
and  A)  during  separate  sessions.  In  the  AVis 
condition  subjects  were  equipped  with  the 
head-mounted  display  and  headphones, 
which  delivered  a  soundscape  updated  in  real 
time  according  to  their  movement  in  the  virtual 
town.  In  the  third  condition,  they  were  asked 
to  navigate  in  a  soundscape  in  the  absence  of 
vision  (A).  The  sounds  were  produced 
through  tracked  binaural  rendering  (HRTF) 
and  were  dependent  upon  the  subject's  move¬ 
ments.  During  all  conditions,  skin  conduc¬ 
tance  was  recorded. 
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